List:Cluster« Previous MessageNext Message »
From:Clint Byrum Date:August 31 2004 8:35pm
Subject:Re: Database Storage Capacity.
View as plain text  
On Tuesday, August 31, 2004, at 01:02 PM, Devananda wrote:

> Faraz Khan wrote:
>> So is it possible at all to setup a terabyte database with Mysql at 
>> the moment?
>> I basically need a DB with 6-10TB storage capacity and ease of growth.
>> I guess I can go with software raid and md devices / nfs but would a 
>> single Mysql server be able to handle that database size?
>> Any help would be appreciated.
>
> Assuming 24GB ram per machine, 2 replicas, you would need ~80 DB nodes 
> to reach 1 TB ... this would be increadibly expensive and seems to me 
> actually not at all what you are looking for.
>
> The MySQL Cluster provides "In Memory" storage that is also supported 
> by all the data being saved to disk (unlike other in memory engines). 
> However, this means that you would have to keep ALL your data in RAM, 
> ALL the time. (Did I hear Mikael say they are thinking of changing 
> this???) For a large DB (10-50GB), this could be a huge benefit, but 
> to one that is 10TB ... it is, as far as I know, impractical at this 
> point to put that into a cluster.
>
> FYI, we just looked into putting a 75-100GB table into the cluster, 
> but figured out that we can house the same table by load balancing 
> traditional InnoDB databases across ~4 servers, rather than across ~16 
> with the cluster.
>

The way I see it.. MySQL cluster is for storing active transactions and 
live state tables. For most things.. customer records.. order history.. 
searchable listings.. replication is a better idea because many times 
you don't have to worry about the high availability or speed of 
writing, or even really the synchronicity of the data. I haven't tried 
it yet, but it seems feasible to just have a database where all of the 
tables are replicated from one master except the cluster tables, which 
live on all servers. This can even be enforced with fine grained user 
permissions that are different on the readonly and master hosts. Of 
course this makes your application more complex.. as it has to decide 
which server to connect to based on what it wants to change.

> Devananda
> Neopets, Inc
>
> -- 
> MySQL Cluster Mailing List
> For list archives: http://lists.mysql.com/cluster
> To unsubscribe:    
> http://lists.mysql.com/cluster?unsub=1
>

Thread
Database Storage Capacity.Faraz Khan30 Aug
  • Re: Database Storage Capacity.Olivier Kaloudoff30 Aug
    • Re: Database Storage Capacity.Mikael Ronström30 Aug
      • Re: Database Storage Capacity.Faraz Khan30 Aug
        • Re: Database Storage Capacity.Olivier Kaloudoff30 Aug
        • Re: Database Storage Capacity.Devananda31 Aug
          • Re: Database Storage Capacity. (addendum)Devananda31 Aug
          • Re: Database Storage Capacity.Clint Byrum31 Aug
          • Re: Database Storage Capacity.Faraz Khan1 Sep
            • Re: Database Storage Capacity.John David Duncan1 Sep
              • Re: Database Storage Capacity.Faraz Khan1 Sep
          • Re: Database Storage Capacity.Mikael Ronström1 Sep
            • Re: Database Storage Capacity.Devananda2 Sep
            • Re: Database Storage Capacity.Douglas K. Fischer4 Sep
      • New TransportSundeep Narravula31 Aug
        • Re: New TransportTomas Ulin31 Aug
          • Re: New TransportMikael Ronström31 Aug
            • Re: New TransportSundeep Narravula31 Aug