Hi all!
I have a fix for the decimal/numeric bug 41582.
As explained in the reference manual [1], decimals are stored using a
binary format that packs nine decimal digits into four bytes, where
any leftover digits require some fraction of four bytes.
In StorageDatabase::getSegmentValue (yes, this method again(!)) we do
a check on the precision, to see if the value is big enough to need
special treatment:
else if (field->precision < 19 && field->scale == 0)
{
int64 number = (signed char) (*ptr++ ^ 0x80);
for (int n = 1; n < length; ++n)
number = (number << 8) | *ptr++;
if (number < 0)
++number;
value->setValue(number);
}
else
{
BigInt bigInt;
ScaledBinary::getBigIntFromBinaryDecimal((const char*)
ptr, field->precision, field->scale, &bigInt);
value->setValue(&bigInt);
}
break;
This test is obviously wrong., what we want to test for is:
else if (field->precision <= 9 && field->scale == 0)
This will fix the problem.
Even better, we can avoid the test all together and treat all decimal
keys as if they *may* have a precision greater than 9. This way there
will be less code and we can avoid a cluttering 'if'. However, this
will result in an additional function call or more (see the else
clause above).
What do you think? Am I allowed to remove the entire 'else if'? :)
Btw, when I am done with the fix for bug#40607, there will be some
more rewrite in this area as well, but that is a different bug ...
/Lars-Erik
[1] - http://dev.mysql.com/doc/refman/6.0/en/precision-math-decimal-changes.html
