List:Falcon Storage Engine« Previous MessageNext Message »
From:Jim Starkey Date:February 18 2009 9:16pm
Subject:Re: Patch for bug#42208
View as plain text  
Vladislav Vaintroub wrote:
> Ann, thanks. 
> I do understand the "pro-double" arguments, however I think that exact
> representation would give us at least some hope (for the future, I know now
> it does not work)  to answer certain SQL queries efficiently using index
> only without the need to read the record.
>
> select name order by  name, age where name like 'A%' and age  > 10
> can be answered just using index on (name,age) (I deliberately ignore MVCC
> and transaction visibility rules for a moment)
>   
Isn't that like saying that ignoring gravity we could fly?
> Also, I suspect inexact representation of integer/decimal breaks "unique" on
> some probably rare cases. If 2 different very big int64 numbers are mapped
> to a single double,  only one of these numbers can be stored in a unique
> index, isn't it so?
>   
No, we fetch the record and get the real value.  Which we would have to 
do anyway to make sure that the version we can see matches the index.

One way around this is to put transaction id in the index.  This has 
loads of problems:

    * Transaction ids aren't persisted across server invocations
    * All versions of a record would need to reflected in the index
    * A record update that didn't change the index would still require
      changing the transaction in the index

Monty planned to do this with Maria, but I don't he even got close to 
scratching the surface of the problems.
>
>   
>> -----Original Message-----
>> From: Ann.Harrison@stripped [mailto:Ann.Harrison@stripped] On Behalf Of
>> Ann W. Harrison
>> Sent: Wednesday, February 18, 2009 8:16 PM
>> To: Vladislav Vaintroub
>> Cc: Kevin.Lewis@stripped; 'Lars-Erik Bjørk'; 'Jim Starkey'; 'FalconDev'
>> Subject: Re: Patch for bug#42208
>>
>> Vladislav Vaintroub wrote:
>>     
>>> And while we're on it why not to fix integer representation;) Doubles
>>>       
>> are
>>     
>>> strange creation by mathematician, exact longlongs would be really
>>>       
>> nice,
>>     
>>> not?
>>>
>>>       
>> Not, I think.  Yes, we could have a separate index representation
>> for numbers declared as integer, but I don't think that buys us
>> much.  Double precision float is precise to 15 digits, so only
>> numbers in the upper range of int64 lose precision when converted.
>> The conversion guarantees that if x > y when represented as
>> int64, x >= y when represented as double.  That's good enough
>> for our indexes - we have to look at the value anyway.
>>
>> But just being good enough and giving a single representation
>> for numbers isn't the best argument for using double precision
>> as opposed to int64 keys.  Nor is the fact that I happen to
>> know how to mangle a double to byte comparable form, and don't
>> have an algorithm at hand to do the same for int64.  No doubt
>> there is one.
>>
>> Double precision allows us to make on-line changes to the
>> length, scale, and type of numbers without rebuilding indexes.
>> Falcon represents decimal/numeric values as binary with a
>> decimal scale factor - meaning that the value is multiplied
>> by 10^<scale> before being stored and divided by 10^<scale>
>> for comparisons with values with different scales.
>>
>> If someone decides after storing lots of GPS data in meters
>> that the actual precision is centimeters, Falcon would allow
>> the field to change from int64 to decimal [20,2] on line
>> instantly.
>>
>> I think that's good.
>>
>> Cheers,
>>
>> Ann
>>     
>
>   


-- 
Jim Starkey
President, NimbusDB, Inc.
978 526-1376

Thread
Patch for bug#42208Lars-Erik Bjørk16 Feb
  • RE: Patch for bug#42208Vladislav Vaintroub16 Feb
    • RE: Patch for bug#42208Vladislav Vaintroub16 Feb
      • Re: Patch for bug#42208Jim Starkey16 Feb
        • RE: Patch for bug#42208Vladislav Vaintroub16 Feb
    • Re: Patch for bug#42208Jim Starkey16 Feb
      • RE: Patch for bug#42208Vladislav Vaintroub16 Feb
      • Re: Patch for bug#42208Lars-Erik Bjørk17 Feb
        • Re: Patch for bug#42208Kevin Lewis17 Feb
          • Re: Patch for bug#42208Ann W. Harrison18 Feb
            • Re: Patch for bug#42208Ann W. Harrison18 Feb
              • Re: Patch for bug#42208Ann W. Harrison18 Feb
                • Re: Patch for bug#42208Kevin Lewis18 Feb
                  • Re: Patch for bug#42208Ann W. Harrison18 Feb
                    • Re: Patch for bug#42208Kevin Lewis18 Feb
          • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
            • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
              • Re: Patch for bug#42208Kevin Lewis18 Feb
                • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
                  • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
              • Re: Patch for bug#42208Jim Starkey18 Feb
                • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
                  • Re: Patch for bug#42208Jim Starkey18 Feb
                    • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
                      • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
            • Re: Patch for bug#42208Ann W. Harrison18 Feb
              • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
                • Re: Patch for bug#42208Jim Starkey18 Feb
                  • Re: Patch for bug#42208Ann W. Harrison18 Feb
                    • Re: Patch for bug#42208Ann W. Harrison18 Feb
                • Re: Patch for bug#42208Ann W. Harrison18 Feb
                  • RE: Patch for bug#42208Vladislav Vaintroub18 Feb
                    • Re: Patch for bug#42208Ann W. Harrison18 Feb
                    • Re: Patch for bug#42208Kevin Lewis18 Feb
                      • Re: Patch for bug#42208Ann W. Harrison18 Feb
      • Re: Patch for bug#42208MARK CALLAGHAN17 Feb
  • Re: Patch for bug#42208Jim Starkey16 Feb