List:Cluster« Previous MessageNext Message »
From:Yves Trudeau Date:September 23 2010 7:24pm
Subject:Re: Performance and Scalability and cluster
View as plain text  
Hi Jeff,
  my answers are embedded.

On Thu, 2010-09-23 at 10:11 -0400, Schweiger, Jeff - ES wrote:
> Hello,
> I have a problem that I am trying to wrap my head around, and I am
> hoping someone in this group can help me.  We have been testing
> cluster for a while now, and we have liked how cluster has performed
> thus far.  However, before we move into deploying cluster, I would
> like to know the answers to a couple of scalability questions.
> On our current system, we have 4 data nodes, and 2 mysqld nodes that
> run about 60,000 queries a minute.  We expect our traffic to increase
> significantly over the next couple of years, and I am look for some
> guides, or any  source of information  that will help answer the
> following questions.
> 1.  How can get a determination of how my cluster configuration is
> performing.  Given my 4 data node, 2 mysqld nodes, 60,000 query a
> minute load, how close am I to impacting performance?

You know, 60k queries/min is not that large for cluster although it
depends on the queries.  How is the load of your servers at 60k/s?  Can
you estimate what will be the first bottleneck? 

> 2.  Is there any benchmark data out there?

Cluster is extremely app dependent so your app is the best benchmark you

> 3.  How do I know when/how to add nodes to the cluster?  For example,
> what is the deference between adding a mysqld node, and a data node,
> and how do I know when it would be necessary to add either?

If you are out of memory space in the data nodes and/or CPU usage is
high there, then it might be an indication that more data nodes are
needed.  If you can design your applications and schema in a way that a
business logic rule hit only one node group, then you'll have a very
scalable application.

About the same for MySQL.  The rule of thumb is more MySQL nodes than
data nodes although since MySQL can now use a pool of connections to the
NDB cluster, it is less true. 

> 4.  Is there any tips/procedures used to measure performance of
> cluster?

It depends what is important for you, throughput and/or latency.  Tuning
for NDB and your application is critical there. 


> Thank for any help you can provide.
> Jeff Schweiger
> jeff.schweiger@stripped
> 44965 Aviation Dr. Suite 400
> Dulles, VA 20166
> Office: (703) 996-2941
> -----Original Message-----
> From: Dan Frincu [mailto:dfrincu@stripped]
> Sent: Monday, July 26, 2010 12:38 PM
> To: Ricardo Soares Guimarães
> Cc: cluster@stripped
> Subject: Re: Converting MyISAM with 300GB to NDB.
> Try script on the 300GB database (it will take some time) to
> get the settings necessary for NDB, however based on the amount of data
> and the limited amount of RAM, unless you seriously upgrade your RAM and
> add several servers in the equation, I don't see how this would be
> achievable with this H/W configuration.
> You might want to check out this link
> Regards,
> Dan.
> Ricardo Soares Guimarães wrote:
> > Hello,
> >
> > I have one database with 300Gb of data, and want to try NDB.
> >
> > That´s possible?
> >
> > I tried several configurations, and keys, but seems that can´t import
> > such data
> >
> > My server has (now) 16Gb of RAM, but soon will have 32.
> >
> > Anyone tried such configuration?
> >
> > I am using mysql-cluster 7.1.x with 64bits linux server.
> >
> > Thanks
> >
> --
> Systems Engineer
> Streamwide Romania
> This e-mail and any files transmitted with it may be proprietary and are intended
> solely for the use of the individual or entity to whom they are addressed. If you have
> received this e-mail in error please notify the sender.
> Please note that any views or opinions presented in this e-mail are solely those of
> the author and do not necessarily represent those of ITT Corporation. The recipient should
> check this e-mail and any attachments for the presence of viruses. ITT accepts no
> liability for any damage caused by any virus transmitted by this e-mail.

Converting MyISAM with 300GB to NDB.Ricardo Soares Guimarães26 Jul
  • Re: Converting MyISAM with 300GB to NDB.Dan Frincu26 Jul
    • Performance and Scalability and clusterJeff - ES Schweiger23 Sep
  • Re: Converting MyISAM with 300GB to NDB.Dan Frincu26 Jul
  • Re: Converting MyISAM with 300GB to NDB.Stewart Smith26 Jul
    • Re: Converting MyISAM with 300GB to NDB.Ricardo Soares Guimarães26 Jul
      • Re: Converting MyISAM with 300GB to NDB.Jason Brooke26 Jul
  • Re: Converting MyISAM with 300GB to NDB.David Ennis26 Jul
    • Re: Converting MyISAM with 300GB to NDB.Ricardo Soares Guimarães27 Jul
  • Re: Performance and Scalability and clusterYves Trudeau23 Sep
    • RE: Performance and Scalability and clusterJeff - ES Schweiger23 Sep
      • Re: Performance and Scalability and clusterAdam Dixon23 Sep
        • RE: Performance and Scalability and clusterAndrew Morgan24 Sep
          • High availability implementationramarovelo28 Sep
            • RE: High availability implementationAndrew Morgan28 Sep
              • Re: High availability implementationMat Keep28 Sep
              • Re: High availability implementationramarovelo28 Sep
      • Re: Performance and Scalability and clusterMikael Ronstrošm24 Sep