Re: rounding/truncation of decimal fields

[Matt Loschert]

On Sun, 13 Apr 2003, Dan Nelson wrote:

> In the last episode (Apr 13), Matt Loschert said:
> > That may be the case, but I would assume (again, I guess incorrectly)
> > that on a given architecture, the following two values would be
> > treated identically.  Is quoting really supposed to change the
> > representation of the number?
>
> Most definitely.
>
> > CREATE TABLE test1 (field1 decimal(12,2));
> > INSERT INTO test1 VALUES ('0.005');
> > INSERT INTO test1 VALUES ( 0.005 );
> > SELECT * FROM test1;
> > +--------+
> > | field1 |
> > +--------+
> > |   0.00 |
> > |   0.01 |
> > +--------+
> > 2 rows in set (0.02 sec)
>
> The first is inserting a string value, which is probably just copied
> directly into the DECIMAL field (with truncation).  The second is
> inserting a floating-point value, which has to have been parsed from
> the original ASCII SQL statement into a float, then converted back into
> ASCII to store into the DECIMAL field.
>
> --
> 	Dan Nelson
> 	dnelson@stripped

Okay, light bulb time.  I get it.

I was assuming that everything was going through a string to float
conversion, and that therefore, everything would see the same float
idiosyncrasies.  However, since the decimal type uses string-based storage
and manipulation, the string form that I was inserting probably never went
through atof() conversion (or something similar).

Thank you to everyone for the information.

- Matt

--
Matt Loschert - Software Engineer       | email: loschert@stripped    |
ServInt Internet Services               | web:   http://www.servint.net/ |
McLean, Virginia USA                    | phone: [phone]          |