Re: Loss of float precision

[Dave Bailey]

On Tue, Apr 1, 2008 at 4:08 PM, Warren Young <mysqlpp@stripped> wrote:

> Jim Wallace wrote:
> >
> > Below is output from a test program (copying the code from MySQL++) that
> > prints out a float with %f and ostringstream with increasing precision.
> > You can see 9 is a better value than 7, but higher than 9 doesn't help.
>
> After reading some of this: http://en.wikipedia.org/wiki/Floating_point
>
> ...I can see going to 8.  The article says "about 7 decimal digits", but
> that just means it's between 7 and 8, closer to 7 in this instance.
>
> What I don't understand is why going to 9 helps.  If we assume that your
> floating point implementation is adhering to the IEEE 754 standard, the
> bits just aren't there to do anything significant with more than 8 digits.
>
> As for double, we may not have to change anything: "about 16 digits"
> means between 15 and 16, closer to 16.
>

This came up a while back with respect to some patches that Google
contributed to MySQL:

http://code.google.com/p/google-mysql-tools/source/browse/trunk/mysql-patches/mysql-5.0.37-patches

If you view the raw file and look for the definition of

Item_func_ieee754_to_string::val_str(String *str)

the comment specifies that 17 digits of precision are needed for double.
It is also true that 9 digits are needed for single precision.  I didn't
believe
it, but I tested it once with every possible value of a single precision
float
and found that there were cases where 8 digits was not enough to support
a lossless conversion from float to string and back.

-dave