List:Cluster« Previous MessageNext Message »
From:Mikael Ronström Date:August 10 2004 9:14am
Subject:Re: memory requirements
View as plain text  
Hi Richard,

2 GB database

Assumptions:
2 replicas
20 million records
2 DB nodes
2 ordered indexes per record
1 primary key hash index (mandatory)

=> 2 GB DataMemory + 600 MB IndexMemory (hash index) + 320 MB 
DataMemory (record overhead) + 170 MB internal node memory +
      500 MB OS, other processes etc + 400 MB (ordered indexes)

As you can see the memory required is
DB size + (Currently only fixed size records, variable sized fields is 
in development)
(25 + PK size)*#no_records + (Primary hash key mandatory for all 
tables, PK_size = 8 bytes if no PK declared)
16*no_records + (Record overhead = handle, checksum, NULL bits, etc)
10*no_records*no_of_ordered_indexes + (Overhead 10 byte per ordered 
index record)
(25 + 16 + PK_size + UK_size)*no_records * no_of_unique_indexes + 
(Overhead for Unique Hash Index)
170MB + (Internal buffers, transaction state in default config)
memory for other processes and OS (normally a set of other processes, 
the OS runs on the machine as well)

Sidenote:
An important note here not yet documented in the MySQL manual is what 
indexes you get.
If you provide a Primary Key it will as default be both a hash index 
and an ordered index. The hash index is mandatory
but the ordered index can be removed by using the syntax USING HASH 
(e.g. primary key (a,b) using hash)
If you provide a unique index it is the same that the hash index part 
is mandatory but as default there will also be an
ordered index. Also here it is possible to remove the ordered index by 
using the same syntax (e.g. unique (a,b) using hash).
Normal indexes declared will always be only ordered indexes.

So 2 machines equipped with 4 GB memory sounds doable, although 5 GB 
wouldn't harm.

If you run MySQL Servers on the same machines these need some amount of 
memory as well.

Disk data is currently developed aimed for MySQL 5.0. It will make it 
possible to store records on disk but fields with indexes will still be 
stored in memory.

Rgrds Mikael

2004-08-10 kl. 04.08 skrev Richard Goh:

>
> --- Mikael_Ronström <mikael@stripped> wrote:
>
>>
>> 2004-08-04 kl. 11.41 skrev Richard Goh:
>>
>>> Hello,
>>>
>>> I'm rather new to mysql clustering.  There are a
>> few
>>> things I'd like to ask:
>>>
>>> From the mysql faq, the recommended RAM is 16G.
>> Is
>>> this clustering being in the memory?
>>>
>>
>> Currently MySQL Cluster is memory-based, yes.
>> 16 GB is recommended but much smaller sizes are
>> possible. Successful
>> experiments
>> have been reported on this list with 264 MByte and
>> 240 Mhz CPU.
>>
>>> From the mysql clustering architecture file, it
>> stated
>>> that it can do such that if a transaction in 1st
>> node
>>> is not successful, then all the rest can be told
>> to
>>> rollback.  Is this something similar to 2 phase
>> commit?
>>>
>>
>> The cluster SW contains a variant of the 2 phase
>> commit protocol. For
>> details on this
>> see my Ph.D thesis that with a little search was
>> still accessible on
>> the Internet at
>> http://citeseer.ist.psu.edu/ronstrom97design.html
>> The transaction protocol is covered in chapter 6.
>>
>> Eventually this will be covered by the MySQL manual.
>
>
> Hi, thanks for the reply!  Was wondering, if this is a
> in-memory database, then how much RAM do I need for
> around 2G of data?  Must be > 2G?
>
> Is there any possibility that it is not going to be
> in-memory?  Because the RAM requirement seems to be
> too high....
>
>
> 		
> __________________________________
> Do you Yahoo!?
> Take Yahoo! Mail with you! Get it on your mobile phone.
> http://mobile.yahoo.com/maildemo
>
> -- 
> MySQL Cluster Mailing List
> For list archives: http://lists.mysql.com/cluster
> To unsubscribe:    
> http://lists.mysql.com/cluster?unsub=1
>
>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com

Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp


Thread
memory requirementsRichard Goh4 Aug
  • Re: memory requirementsMikael Ronström4 Aug
    • Re: memory requirementsRichard Goh10 Aug
      • Re: memory requirementsMikael Ronström10 Aug