bk commit into 6.0 tree (sergefp:1.2658) BUG#32191

[Sergey Petrunia]

Below is the list of changes that have just been committed into a local
6.0 repository of psergey. When psergey does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet@stripped, 2007-11-14 21:39:26+03:00, sergefp@stripped +2 -0
  BUG#32191: Memory overrun when using join buffering for falcon table with a blob:
  The problem: Falcon didn't follow the established convention that
    "If a blob column has NULL value, then its length and blob data 
     pointer must be set to 0".
  The fix:
  - Made falcon follow the convention
  - Documented it
  There is no testcase as this is buffer overrun and small repeatable testcase
  cannot be made.

  sql/handler.h@stripped, 2007-11-14 21:39:21+03:00, sergefp@stripped +43 -0
    BUG#32191: Memory overrun when using join buffering for falcon table with a blob
    - Added a comment about TableRecordFormat convention. The part that 
      actually relates to this bug is that Falcon didn't follow the 
      "If a blob column has NULL value, then its length and blob data 
       pointer must be set to 0" convention.

  storage/falcon/ha_falcon.cpp@stripped, 2007-11-14 21:39:21+03:00, sergefp@stripped +4 -0
    BUG#32191: Memory overrun when using join buffering for falcon table with a blob
     - Make Falcon, like all other engines, bzero blob length/pointer when 
       returning NULL blob value

diff -Nrup a/sql/handler.h b/sql/handler.h
--- a/sql/handler.h	2007-10-09 03:48:44 +04:00
+++ b/sql/handler.h	2007-11-14 21:39:21 +03:00
@@ -1157,6 +1157,49 @@ uint calculate_key_len(TABLE *, uint, co
   The handler class is the interface for dynamically loadable
   storage engines. Do not add ifdefs and take care when adding or
   changing virtual functions to avoid vtable confusion
+
+  Functions in this class accept and return table columns data. Two data
+  representation formats are used:
+  1. TableRecordFormat - Used to pass [partial] table records to/from
+     storage engine
+
+  2. KeyTupleFormat - used to pass index search tuples (aka "keys") to
+     storage engine. See opt_range.cc for description of this format.
+
+  TableRecordFormat
+  =================
+  [Warning: this description is work in progress and may be incomplete]
+  The table record is stored in a fixed-size buffer:
+   
+    record: null_bytes, column1_data, column2_data, ...
+  
+  The offsets of the parts of the buffer are also fixed: every column has 
+  an offset to its column{i}_data, and if it is nullable it also has its own
+  bit in null_bytes. 
+
+  The record buffer only includes data about columns that are marked in the
+  relevant column set (table->read_set and/or table->write_set, depending on
+  the situation). 
+  <not-sure>It could be that it is required that null bits of non-present
+  columns are set to 1</not-sure>
+
+  VARIOUS EXCEPTIONS AND SPECIAL CASES
+
+  f the table has no nullable columns, then null_bytes is still 
+  present, its length is one byte <not-sure> which must be set to 0xFF 
+  at all times. </not-sure>
+  
+  If the table has columns of type BIT, then certain bits from those columns
+  may be stored in null_bytes as well. Grep around for Field_bit for
+  details.
+
+  For blob columns (see Field_blob), the record buffer stores length of the 
+  data, following by memory pointer to the blob data. The pointer is owned 
+  by the storage engine and is valid until the next operation.
+
+  If a blob column has NULL value, then its length and blob data pointer
+  must be set to 0.
+
  */
 
 class handler :public Sql_alloc
diff -Nrup a/storage/falcon/ha_falcon.cpp b/storage/falcon/ha_falcon.cpp
--- a/storage/falcon/ha_falcon.cpp	2007-10-10 09:43:44 +04:00
+++ b/storage/falcon/ha_falcon.cpp	2007-11-14 21:39:21 +03:00
@@ -2166,7 +2166,11 @@ void StorageInterface::decodeRecord(ucha
 			field->move_field_offset(ptrDiff);
 
 		if (dataStream->type == edsTypeNull || !bitmap_is_set(table->read_set, field->field_index))
+			{
 			field->set_null();
+                        if (field->real_type() == MYSQL_TYPE_BLOB)
+				field->reset();
+			}
 		else
 			{
 			field->set_notnull();