List:General Discussion« Previous MessageNext Message »
From:Dan Nelson Date:October 27 1999 3:26pm
Subject:Re: Problems with floor
View as plain text  
In the last episode (Oct 27), sinisa@stripped said:
> There were questions on the list regarding a query like this:
> 
> select floor(pow(2,63));
> 
> returning negative result of -9223372036854775808.
> 
> After some small analysis, I do have a definite answer for you.
> 
> According to a standard, floor has to return an integer. MySQL uses
> long long type for it, because it is the largest signed integer type
> available.

Floor has to return an integral value, but it doesn't guarantee that
that value will fit in an integer-type variable.
 
> So, as :
> 
> pow(2,63) = (double) 9223372036854775808.000000;
> 
> floor( (double) 9223372036854775808.000000) = 9223372036854775808.000000;
> 
> Largest long long number is 9223372036854775807 !!!!!!!
> 
> Therefore (long long) 9223372036854775808.000000 =  -9223372036854775808;
> 
> Unsigned long long could not be uses, as it has to be signed integer
> !!qs
> 
> So, everything is as it should be !

But, according to the X/Open manpage for floor() at
http://www.opengroup.org/onlinepubs/007908799/xsh/floor.html :

     APPLICATION USAGE

     The integral value returned by floor() as a double might not be
     expressible as an int or long int. The return value should be
     tested before assigning it to an integer type to avoid the
     undefined results of an integer overflow.

Mysql should either return some kind of error, or keep the result in
double format internally, or use fixed-point math..

-- 
	Dan Nelson
	dnelson@stripped
Thread
Re: Problems with floorsinisa27 Oct
  • Re: Problems with floorDan Nelson27 Oct
    • Re: Problems with floorsinisa27 Oct
      • Re: Problems with floorDan Nelson27 Oct
        • Re: Problems with floorMichael Widenius15 Nov
  • Re: Problems with floorJeremy Cole28 Oct
    • Re: Problems with floorsinisa28 Oct