List:General Discussion« Previous MessageNext Message »
From:Jonas Oreland Date:May 26 2004 5:23am
Subject:Re: Large database with High Availability - how to do it?
View as plain text  
Jochem van Dieten wrote:
> Daniel Ek wrote:
> 
>> Jonas Oreland wrote:
>>
>>> Daniel Ek wrote:
>>>
>>>>
>>>> I today have a database using 20GB storage, which actually means 
>>>> that I should have at least 30GB (40GB recommended) ram using Mysql 
>>>> cluster.
>>>
>>>
>>> This could be 40 boxes with 1G ram each.
> 
> 
> But doesn't that increase the locking conflicts significantly? It is my 
> understanding that 2PC locking conflicts scale with the third power of 
> the number of nodes:
> http://citeseer.ist.psu.edu/gray96danger.html

1) We use "primay copy", so locking is done in a predefined order per row.
2) The data is partitioned and not fully replicated
3) The partitioning is transparent to a MySQL user, but there are means 
of influencing the partitioning (still not exposed in sql :-()

With this we achived good scalability,

but depending on the usage profile, ethernet can put a limit to the 
scalability (with our current "latest-and-greatest" solutions).

This is why we also work with different interconnects such as sci.

> 
> 
>>> Btw: We're also working on adding "disk data", i.e. data that does
>>> not have to reside in ram all the time. This will however probably 
>>> not be finished until the end of this year.
> 
> 
> How about a synchronous transaction log? 99.999% availability is nice 
> for the PHB, but my concern is what happens when the 0.001% hits the 
> fan, the whole cluster goes down and the transaction log has not yet 
> been written to disk.

The database does continuous checkpointing to disk,
but does not flush the log on commit.
So in the unlikely event of system failure, it will restart from the 
lastest checkpoint. The checkpointing interval is configurable (with a 
default of 1 sec)

Disk durable commits is on the release plan (somewhere in time)

> 
> Does MySQL Cluster meet the D of ACID?
> 

In the sence, that each commit is made on multiple boxes.

Note also that the no of copies is configurable, between 1 & 4,
so with 4 copies, the data is commited in 4 boxes before ack-ed to the 
users.

Regards,
	Jonas Oreland
Thread
Large database with High Availability - how to do it?Daniel Ek25 May
  • Re: Large database with High Availability - how to do it?Jonas Oreland26 May
Re: Large database with High Availability - how to do it?Daniel Ek25 May
  • Re: Large database with High Availability - how to do it?Jochem van Dieten25 May
    • Re: Large database with High Availability - how to do it?Jonas Oreland26 May
  • Re: Large database with High Availability - how to do it?Jonas Oreland26 May