Hi Oystein,
The key_info->key_length is the sum of lengths of all key parts.
In the code part that you're fixing key_length is calculated as
key_length= sum(field->pack_length()) + (#_of_null_fields +7)/8
(# + 7)/8 reflects the way nulls bits are stored within a table, they are packed
in bytes. But key handling code assumes nulls to be stored in a byte per field.
Because of this when cache is allowed the cmp_buffer_with_ref() doesn't copy
enough data to see the difference. And we see the bug.
With your addition it becomes
key_length= sum(field->pack_length()) + (#_of_null_fields +7)/8 +
#_of_null_fields * HA_KEY_NULL_LENGTH +
#_of_blob_or_vchar_fields * HA_KEY_BLOB_LENGTH
As you can see nulls are taken into account twice. It's not a big deal by itself
since it would lead to just allocating bigger key buffer.
But anyway, I suggest you to make the calculation more clear by setting the
key_length to 0 and then to sum all key part lengths. Te same as the group part
in the create_tmp_table do.
Regards, Evgen.
Oystein.Grovlen@stripped wrote:
> #At file:///home/oysteing/mysql/mysql-6.0-codebase-bugfixing-3/ based on
> revid:sergey.glukhov@stripped
>
> 3672 oystein.grovlen@stripped 2009-10-26
> Bug#46548 IN-subqueries return 0 rows with materialization=on
>
