List:Cluster« Previous MessageNext Message »
From:John David Duncan Date:September 1 2004 5:38am
Subject:Re: Database Storage Capacity.
View as plain text  
Using NFS for that amount of data is probably not a great
idea.  If you're running Linux, you might want to look at
the Lustre filesystem -- www.clusterfs.com or www.lustre.org.
It could provide high availability for you at the filesystem
level.

As far MySQL -- it will certainly run with that much data.
But the more RAM you have, the greater portion of your indexes
you can have cached in memory, and the better your performance
will be.

- JD


On Aug 31, 2004, at 10:15 PM, Faraz Khan wrote:

> Assuming I dont put this into a cluster at all.. How else can I 
> acheieve a
> Mysql Database thats AS big? I basically am asking two questions:
>
> 1. HOW do i put together a mysql solution that has a storage capacity 
> of 10Tb?
> Do I have to opt for storage devices or can I build one of my own using
> Raid / nfs?
>
> 2. The question that has been answered-- how to provide high 
> availability on
> it.
>
>
> I know this may be deviating from the subject a little bit. But I was
> wondering if i could get some feedback on this.
>
>
> On Wed 1/09/'04 1:02 am, Devananda wrote:
>> Faraz Khan wrote:
>>> So is it possible at all to setup a terabyte database with Mysql at 
>>> the
>>> moment?
>>>
>>> I basically need a DB with 6-10TB storage capacity and ease of 
>>> growth.
>>> I guess I can go with software raid and md devices / nfs but would a
>>> single Mysql server be able to handle that database size?
>>>
>>>
>>> Any help would be appreciated.
>>
>> Assuming 24GB ram per machine, 2 replicas, you would need ~80 DB nodes
>> to reach 1 TB ... this would be increadibly expensive and seems to me
>> actually not at all what you are looking for.
>>
>> The MySQL Cluster provides "In Memory" storage that is also supported 
>> by
>> all the data being saved to disk (unlike other in memory engines).
>> However, this means that you would have to keep ALL your data in RAM,
>> ALL the time. (Did I hear Mikael say they are thinking of changing
>> this???) For a large DB (10-50GB), this could be a huge benefit, but 
>> to
>> one that is 10TB ... it is, as far as I know, impractical at this 
>> point
>> to put that into a cluster.
>>
>> FYI, we just looked into putting a 75-100GB table into the cluster, 
>> but
>> figured out that we can house the same table by load balancing
>> traditional InnoDB databases across ~4 servers, rather than across ~16
>> with the cluster.
>>
>> Devananda
>> Neopets, Inc
>
> -- 
> Faraz Khan
> Busines Development Manager
> Enterprise Solutions
> Inbox Business Technologies (Pvt.) Ltd.
> 111-551-551
> faraz.khan@stripped
>
> -- 
> MySQL Cluster Mailing List
> For list archives: http://lists.mysql.com/cluster
> To unsubscribe:    http://lists.mysql.com/cluster?unsub=1
>

Thread
Database Storage Capacity.Faraz Khan30 Aug
  • Re: Database Storage Capacity.Olivier Kaloudoff30 Aug
    • Re: Database Storage Capacity.Mikael Ronström30 Aug
      • Re: Database Storage Capacity.Faraz Khan30 Aug
        • Re: Database Storage Capacity.Olivier Kaloudoff30 Aug
        • Re: Database Storage Capacity.Devananda31 Aug
          • Re: Database Storage Capacity. (addendum)Devananda31 Aug
          • Re: Database Storage Capacity.Clint Byrum31 Aug
          • Re: Database Storage Capacity.Faraz Khan1 Sep
            • Re: Database Storage Capacity.John David Duncan1 Sep
              • Re: Database Storage Capacity.Faraz Khan1 Sep
          • Re: Database Storage Capacity.Mikael Ronström1 Sep
            • Re: Database Storage Capacity.Devananda2 Sep
            • Re: Database Storage Capacity.Douglas K. Fischer4 Sep
      • New TransportSundeep Narravula31 Aug
        • Re: New TransportTomas Ulin31 Aug
          • Re: New TransportMikael Ronström31 Aug
            • Re: New TransportSundeep Narravula31 Aug