List:MySQL++« Previous MessageNext Message »
From:Jim Wallace Date:April 1 2008 6:49pm
Subject:Loss of float precision
View as plain text  
We had problems with numbers in the 1 million range getting truncated
using MySQL++, but in C++ the values were not getting truncated.
Something was getting lost in the translation  Doing some testing I
found out the following:


		float f = 1213567.75;
		query.reset();
		query << "insert into floattest values (" << f << ")";
// uses STL's vanilla shift conversion
// generates insert into floattest values (1.21357e+006)
		query.execute();
		query.reset();
		query << "insert into floattest values (%0)";
		query.parse();
		f = 1213568.75;
		query.def[0U] =  f;
// uses MySQL++ ostringstream conversion, with only a precision of 7
// generates insert into floattest values (1213569)
		query.execute();

If I change the precision used by MySQL++ to 9 this was generated:
		query.def[0U] =  f;
insert into floattest values (1213568.75)

Casting to double helped, too.
		query.def[0U] =  (double)f;
insert into floattest values (1213568.75)


Below is output from a test program (copying the code from MySQL++) that
prints out a float with %f and ostringstream with increasing precision.
You can see 9 is a better value than 7, but higher than 9 doesn't help.
Without digressing into a discussion of best datatypes, it seems like
the fix is to use a precision of 9 so the C++ values get into the
database unchanged.  Comments?  I haven't investigated SQLSS will it do
the same thing?

 float f = 1213567.75;

Pr printf %f         ostringstream with precision of Pr
 1 1213567.750000 == 1e+006
 2 1213567.750000 == 1.2e+006
 3 1213567.750000 == 1.21e+006
 4 1213567.750000 == 1.214e+006
 5 1213567.750000 == 1.2136e+006
 6 1213567.750000 == 1.21357e+006
 7 1213567.750000 == 1213568
 8 1213567.750000 == 1213567.8
 9 1213567.750000 == 1213567.75
10 1213567.750000 == 1213567.75
11 1213567.750000 == 1213567.75
12 1213567.750000 == 1213567.75
13 1213567.750000 == 1213567.75
14 1213567.750000 == 1213567.75
15 1213567.750000 == 1213567.75
Thread
RELEASE: v3.0.1Warren Young25 Mar
  • Loss of float precisionJim Wallace1 Apr
    • Re: Loss of float precisionWarren Young2 Apr
      • Re: Loss of float precisionDave Bailey2 Apr
        • Re: Loss of float precisionWarren Young2 Apr
          • RE: Loss of float precisionJim Wallace2 Apr
            • Re: Loss of float precisionWarren Young2 Apr