List:MySQL++« Previous MessageNext Message »
From:Warren Young Date:April 1 2008 11:08pm
Subject:Re: Loss of float precision
View as plain text  
Jim Wallace wrote:
> 
> Below is output from a test program (copying the code from MySQL++) that
> prints out a float with %f and ostringstream with increasing precision.
> You can see 9 is a better value than 7, but higher than 9 doesn't help.

After reading some of this: http://en.wikipedia.org/wiki/Floating_point

...I can see going to 8.  The article says "about 7 decimal digits", but 
that just means it's between 7 and 8, closer to 7 in this instance.

What I don't understand is why going to 9 helps.  If we assume that your 
floating point implementation is adhering to the IEEE 754 standard, the 
bits just aren't there to do anything significant with more than 8 digits.

As for double, we may not have to change anything: "about 16 digits" 
means between 15 and 16, closer to 16.

> I haven't investigated SQLSS will it do
> the same thing?

It uses the same underlying conversion mechanism as the rest of the 
library in v3: SQLTypeAdapter.
Thread
RELEASE: v3.0.1Warren Young25 Mar
  • Loss of float precisionJim Wallace1 Apr
    • Re: Loss of float precisionWarren Young2 Apr
      • Re: Loss of float precisionDave Bailey2 Apr
        • Re: Loss of float precisionWarren Young2 Apr
          • RE: Loss of float precisionJim Wallace2 Apr
            • Re: Loss of float precisionWarren Young2 Apr