List:Cluster« Previous MessageNext Message »
From:Devananda Date:August 20 2004 11:34pm
Subject:Re: Data persistency
View as plain text  
Mikael Ronström wrote:

> 64 bytes for the VARCHAR(20) seems correct.
> Hash Index for 64 byte PK actually is 33 + 64 = 107 bytes per key 
> (there is a long key feature that kicks in above 32 bytes that has an 
> 8 byte overhead)
>  => This is obviously what causes the 3-4M limit due to the IndexMemory.
>
> 3 * 64 + 4* 4 + (3 * 100 + 4) + 16 = 524 bytes per record in DataMemory
> => 61 records per page => 75M records ~40GByte (Thus ~10 GByte 
> DataMemory per node)
>
> 107 * 75 M * 2 replicas = 16 GB => 4 GByte IndexMemory per node
>
> => 14 GByte memory per machine.
>
> Thus 16 GByte machines should do the trick hopefully.
>
>> ERROR 1064 (42000): You have an error in your SQL syntax; check the 
>> manual that corresponds to your MySQL server version for the right 
>> syntax to use near 'USING HASH
>>
>> Assuming I can drop the ordered index and that UTF8 requires tripple 
>> the space, there will be 601 bytes/per record => 54 records/page => 
>> 54k records/ 32M => 75M records ~ 45GB. Then with 8 nodes and 2 
>> replicas (thus 4 pairs), this would require a minimum of 12GB 
>> DataMemory per machine. Is this correct?
>>
>
> Currently the ordered index would not work with UTF8 since it requires 
> the ndb kernel to use the UTF8 compare routines which is still on the 
> TODO list.
> Using the hash index is ok as long as there are exact matches. The NDB 
> kernel doesn't handle the data, it only puts it into memory and later 
> retrieves it
> from memory so the rest should be ok.
>
> Rgrds Mikael
>
>  

This is really really helpful! Now I know how large of a machine we will 
have to purchase to set up this cluster :)

Thanks again!
Devananda
Neoepts, Inc
Thread
Data persistencyRichard Goh19 Aug
  • Re: Data persistencyMikael Ronström19 Aug
    • Re: Data persistencyDevananda19 Aug
      • Re: Data persistencyMikael Ronström20 Aug
        • Re: Data persistencyDevananda20 Aug
          • Re: Data persistencyDevananda20 Aug
          • Re: Data persistencyMikael Ronström21 Aug
            • Re: Data persistencyDevananda21 Aug
            • Re: Data persistencyClint Byrum21 Aug
              • Re: Data persistencyMikael Ronström21 Aug