List:General Discussion« Previous MessageNext Message »
From:Matt Loschert Date:April 13 2003 4:00am
Subject:Re: rounding/truncation of decimal fields
View as plain text  
On Sat, 12 Apr 2003, Doug Thompson wrote:

> On Sat, 12 Apr 2003 10:41:01 -0400 (EDT), Matt Loschert wrote:
>
> >Interesting.  I had never seen this page of the manual before.  However,
> >you appear to be correct.  I don't see an applicable reference there
> >either.
> >
> >I think I am going to re-post my message to the bugs list since no one has
> >shown me yet that this is definitely expected behavior.
> >
>
> Huh?!?
>
> Here are the first two sentences from Appendix A.5.7.
>
> <quote>
> floating-point numbers cause confusion sometimes, because these numbers
> are not stored as exact values inside computer architecture. What one
> can see on the screen usually is not the exact value of the number.
>
> Field types FLOAT, DOUBLE and DECIMAL are such.
> <end quote>
>
> Seems to me that's rather explicit that what you are seeing is expected behavior.

That may be the case, but I would assume (again, I guess incorrectly) that
on a given architecture, the following two values would be treated
identically.  Is quoting really supposed to change the representation of
the number?

CREATE TABLE test1 (field1 decimal(12,2));
INSERT INTO test1 VALUES ('0.005');
INSERT INTO test1 VALUES ( 0.005 );
SELECT * FROM test1;
+--------+
| field1 |
+--------+
|   0.00 |
|   0.01 |
+--------+
2 rows in set (0.02 sec)

- Matt

--
Matt Loschert - Software Engineer       | email: loschert@stripped    |
ServInt Internet Services               | web:   http://www.servint.net/ |
McLean, Virginia USA                    | phone: (703) 847-1381          |
Thread
rounding/truncation of decimal fieldsMatt Loschert10 Apr
  • Re: rounding/truncation of decimal fieldsgerald_clark11 Apr
RE: rounding/truncation of decimal fieldsClifton R [CC] Jones11 Apr
  • Re: rounding/truncation of decimal fieldsgerald_clark11 Apr
    • Re: rounding/truncation of decimal fieldsMatt Loschert11 Apr
      • Re: rounding/truncation of decimal fieldsDoug Thompson11 Apr
        • Re: rounding/truncation of decimal fieldsMatt Loschert12 Apr
          • Re: rounding/truncation of decimal fieldsDoug Thompson12 Apr
            • More on enumMartin Gainty12 Apr
            • Re: rounding/truncation of decimal fieldsMatt Loschert13 Apr
              • Re: rounding/truncation of decimal fieldsDoug Thompson13 Apr
              • Re: rounding/truncation of decimal fieldsDan Nelson13 Apr
                • Re: rounding/truncation of decimal fieldsMatt Loschert14 Apr
Re: rounding/truncation of decimal fieldsMoon79611 Apr