From: Karen Abgarian Date: March 30 2012 8:30pm Subject: Re: HA & Scalability w MySQL + SAN + VMWare: Architecture Suggestion Wanted List-Archive: http://lists.mysql.com/mysql/227080 Message-Id: MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi,=20 First, it is kind of funny to advise on something that is unknown. The = devil of such systems is in details. A small detail might cancel the = whole big idea of using, say, sharing, clustering, etc. So any = discussion on this will be quite general and can only be applied to your = project with a lot of validation.=20 Second, different folks have very different things in mind when they say = magic words like 'scalability' or 'high availability'. For example, = 'high availability' could mean such things as: - the way to bring back resource online shall be available at the time = the user (application) discovers the failure and can take reasonable = time to be made available;=20 - the backup resource shall be available at the time the user = (application) discovers the failure and requires no preparation;=20 - the user (application) shall not be dependent on the detection and = resolution mechanisms of the underlying technology stack.=20 Having said that, the idea to provide HA with VMWare and node reboots = qualifies this as a project with "eventual" availability as described in = the first statement above. It is qualified as eventual because the = backup system is not verified as being available, and the VMWare might = fail to bring the node online, which is not controlled by no software.=20= The 'scalability', for example, could also have a number of different = meanings:=20 - the system shall allow placing new growth data on the additional = nodes; =20 - the system shall dynamically place the new growth data on the = additional nodes;=20 - the system shall allow redistributing access to the data among the = available nodes;=20 - the system shall load balance access to the data among the available = nodes;=20 - the system shall load balance the access and data across all available = nodes and keep all nodes equally loaded.=20 Now, to the question of one big instance and multiple instances. = People usually get stuck with one big instance if they just can't break = it into smaller instances. They usually can't break into multiple = instances if such instances would have interdependencies which cannot be = removed by changing the app design (or design cannot be easily changed). = Such interdependencies may randomly alter your scalability and = availability. For example, if you have some critical data on just one = node out of dozen, the availability of the whole system will be impacted = if that node is down. =20 =85. I ran out of time. But on these subjects, the chatting could go on = for years. You may want to clearer explain what you are trying to do. = It could make discussion more focused. Peace, Karen Abgarian. On Mar 29, 2012, at 6:23 PM, Wes Modes wrote: > First, thank you in advance for good solid suggestions you can offer. = I > suppose someone has already asked this, but perhaps you will view it = as > a fun challenge to meet my many criteria with your suggested MySQL > architecture. >=20 > I am working at a University on a high-profile database driven project > that we expect to be slammed within the first few months. Since this = is > a new project and one that we expect to be popular, we don't know what > kind of usage to expect, but we want to be prepared. Therefore, we are > building in extra capacity. >=20 > Our top goals are scalability and high availability, provided we hope > through multiple MySQL nodes and VMWare functionality. I've been > surprised that there are not more MySQL architects trying to meet = these > high-level goals using virtualization and shared storage (or at least > they do not seem to be writing about it). >=20 > I've looked at replication, multi-mastering, DRBD, clustering, > partitioning, and sharding. >=20 > Here's what we got, and some of our constraints: >=20 > * We are concerned that One Big Database instance won't be enough to > handle all of the queries, plus it is a single point of failure. > Therefore, multiple nodes are desirable. >=20 > * With the primary application that will be using the database, writes > and reads cannot be split off from each other. This limitation alone, > rules out replication, MMM, and a few other solutions. >=20 > * We do not expect to be especially write-heavy. >=20 > * We have shared storage in the form of an iSCSI SAN. We'd like to > leverage the shared storage, if possible. >=20 > * We have VMWare HA which already monitors hosts and brings them up > within minutes elsewhere if we lose a host. So some of the suggested = HA > solutions are redundant. >=20 > * We expect to have another instance of our system running in the = Amazon > cloud for the first few months while the traffic is high, so we may = take > advantage of RDS, though an exact duplicate of our local system will > save us development work. >=20 > Thanks for any advice you can give. >=20 > Wes Modes >=20 > --=20 > Wes Modes > Systems Designer, Developer, and Administrator > University Library ITS > University of California, Santa Cruz >=20 >=20 > --=20 > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/mysql >=20