List:General Discussion« Previous MessageNext Message »
From:Karen Abgarian Date:March 30 2012 8:30pm
Subject:Re: HA & Scalability w MySQL + SAN + VMWare: Architecture Suggestion
Wanted
View as plain text  
Hi, 

First, it is kind of funny to advise on something that is unknown.   The devil of such
systems is in details.   A small detail might cancel the whole big idea of using, say,
sharing, clustering, etc.    So any discussion on this will be quite general and can only
be applied to your project with a lot of validation. 

Second, different folks have very different things in mind when they say magic words like
'scalability' or 'high availability'.    For example, 'high availability' could mean such
things as:

- the way to bring back resource online shall be available at the time the user
(application) discovers the failure and can take reasonable time to be made available; 
- the backup resource shall be available at the time the user (application) discovers the
failure and requires no preparation; 
- the user (application) shall not be dependent on the detection and resolution mechanisms
of the underlying technology stack. 

Having said that, the idea to provide HA with VMWare and node reboots qualifies this as a
project with "eventual" availability as described in the first statement above.   It is
qualified as eventual because the backup system is not verified as being available, and
the VMWare might fail to bring the node online, which is not controlled by no software. 

The 'scalability', for example, could also have a number of different meanings: 

- the system shall allow placing new growth data on the additional nodes;  
- the system shall dynamically place the new growth data on the additional nodes; 
- the system shall allow redistributing access to the data among the available nodes; 
- the system shall load balance access to the data among the available nodes; 
- the system shall load balance the access and data across all available nodes and keep
all nodes equally loaded. 

Now, to the question of one big instance and multiple instances.   People usually get
stuck with one big instance if they just can't break it into smaller instances.    They
usually can't break into multiple instances if such instances would have
interdependencies which cannot be removed by changing the app design (or design cannot be
easily changed).    Such interdependencies may randomly alter your scalability and
availability.   For example, if you have some critical data on just one node out of
dozen, the availability of the whole system will be impacted if that node is down.   

….

I ran out of time.    But on these subjects, the chatting could go on for years.   You may
want to clearer explain what you are trying to do.  It could make discussion more focused.

Peace,
Karen Abgarian.


On Mar 29, 2012, at 6:23 PM, Wes Modes wrote:

> First, thank you in advance for good solid suggestions you can offer. I
> suppose someone has already asked this, but perhaps you will view it as
> a fun challenge to meet my many criteria with your suggested MySQL
> architecture.
> 
> I am working at a University on a high-profile database driven project
> that we expect to be slammed within the first few months. Since this is
> a new project and one that we expect to be popular, we don't know what
> kind of usage to expect, but we want to be prepared. Therefore, we are
> building in extra capacity.
> 
> Our top goals are scalability and high availability, provided we hope
> through multiple MySQL nodes and VMWare functionality. I've been
> surprised that there are not more MySQL architects trying to meet these
> high-level goals using virtualization and shared storage (or at least
> they do not seem to be writing about it).
> 
> I've looked at replication, multi-mastering, DRBD, clustering,
> partitioning, and sharding.
> 
> Here's what we got, and some of our constraints:
> 
> * We are concerned that One Big Database instance won't be enough to
> handle all of the queries, plus it is a single point of failure.
> Therefore, multiple nodes are desirable.
> 
> * With the primary application that will be using the database, writes
> and reads cannot be split off from each other. This limitation alone,
> rules out replication, MMM, and a few other solutions.
> 
> * We do not expect to be especially write-heavy.
> 
> * We have shared storage in the form of an iSCSI SAN. We'd like to
> leverage the shared storage, if possible.
> 
> * We have VMWare HA which already monitors hosts and brings them up
> within minutes elsewhere if we lose a host. So some of the suggested HA
> solutions are redundant.
> 
> * We expect to have another instance of our system running in the Amazon
> cloud for the first few months while the traffic is high, so we may take
> advantage of RDS, though an exact duplicate of our local system will
> save us development work.
> 
> Thanks for any advice you can give.
> 
> Wes Modes
> 
> -- 
> Wes Modes
> Systems Designer, Developer, and Administrator
> University Library ITS
> University of California, Santa Cruz
> 
> 
> -- 
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe:    http://lists.mysql.com/mysql
> 

Thread
HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes30 Mar
  • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedRick James30 Mar
  • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedKaren Abgarian30 Mar
  • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedshawn green30 Mar
    • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes2 Apr
      • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedRick James2 Apr
      • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedshawn green2 Apr
        • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes2 Apr
          • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture Suggestion WantedTim Dunphy3 Apr
          • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedIan3 Apr
            • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture Suggestion WantedJohan De Meersman3 Apr
            • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes3 Apr
            • Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes4 Apr
        • Re: Replication rings/maatkit (was: HA & Scalability w MySQL + SAN+ VMWare)Wes Modes3 Apr
          • Re: Replication rings/maatkit (was: HA & Scalability w MySQL + SAN + VMWare)Eric Bergen3 Apr
Re: HA & Scalability w MySQL + SAN + VMWare: Architecture SuggestionWantedWes Modes4 Apr