Thanks for the response... One follow up question:
>You know, 60k queries/min is not that large for cluster although it
>depends on the queries. How is the load of your servers at 60k/s? Can
>you estimate what will be the first bottleneck?
I guess that is part of my problem... I would like to get an idea how many queries
cluster *can* handle. At my current query rate, the load on my servers is very low. I
know performance is highly variable and is app dependant, hardware dependant and query
type dependant. For example, currently I am running on old Sun 2000's, and know I will
get a boost if I go to a newer Intel multi core platform. But I think I have tuned by
app and cluster to perform well.
So given a 4 data node 2 mysqld cluster, can I expect 500,000 queries per second, can I
expect more? I would love to know what type of performance people have seen.
Thanks.
Jeff Schweiger
jeff.schweiger@stripped
44965 Aviation Dr. Suite 400
Dulles, VA 20166
Office: (703) 996-2941
-----Original Message-----
From: Yves Trudeau [mailto:y.trudeau@stripped]
Sent: Thursday, September 23, 2010 3:24 PM
To: Schweiger, Jeff - ES
Cc: cluster@stripped
Subject: Re: Performance and Scalability and cluster
Hi Jeff,
my answers are embedded.
On Thu, 2010-09-23 at 10:11 -0400, Schweiger, Jeff - ES wrote:
> Hello,
>
> I have a problem that I am trying to wrap my head around, and I am
> hoping someone in this group can help me. We have been testing
> cluster for a while now, and we have liked how cluster has performed
> thus far. However, before we move into deploying cluster, I would
> like to know the answers to a couple of scalability questions.
>
> On our current system, we have 4 data nodes, and 2 mysqld nodes that
> run about 60,000 queries a minute. We expect our traffic to increase
> significantly over the next couple of years, and I am look for some
> guides, or any source of information that will help answer the
> following questions.
>
> 1. How can get a determination of how my cluster configuration is
> performing. Given my 4 data node, 2 mysqld nodes, 60,000 query a
> minute load, how close am I to impacting performance?
You know, 60k queries/min is not that large for cluster although it
depends on the queries. How is the load of your servers at 60k/s? Can
you estimate what will be the first bottleneck?
> 2. Is there any benchmark data out there?
Cluster is extremely app dependent so your app is the best benchmark you
have.
> 3. How do I know when/how to add nodes to the cluster? For example,
> what is the deference between adding a mysqld node, and a data node,
> and how do I know when it would be necessary to add either?
If you are out of memory space in the data nodes and/or CPU usage is
high there, then it might be an indication that more data nodes are
needed. If you can design your applications and schema in a way that a
business logic rule hit only one node group, then you'll have a very
scalable application.
About the same for MySQL. The rule of thumb is more MySQL nodes than
data nodes although since MySQL can now use a pool of connections to the
NDB cluster, it is less true.
> 4. Is there any tips/procedures used to measure performance of
> cluster?
It depends what is important for you, throughput and/or latency. Tuning
for NDB and your application is critical there.
Regards,
Yves
>
> Thank for any help you can provide.
>
> Jeff Schweiger
> jeff.schweiger@stripped
> 44965 Aviation Dr. Suite 400
> Dulles, VA 20166
> Office: (703) 996-2941
>
> -----Original Message-----
> From: Dan Frincu [mailto:dfrincu@stripped]
> Sent: Monday, July 26, 2010 12:38 PM
> To: Ricardo Soares Guimarães
> Cc: cluster@stripped
> Subject: Re: Converting MyISAM with 300GB to NDB.
>
> Try ndb_size.pl script on the 300GB database (it will take some time) to
> get the settings necessary for NDB, however based on the amount of data
> and the limited amount of RAM, unless you seriously upgrade your RAM and
> add several servers in the equation, I don't see how this would be
> achievable with this H/W configuration.
>
> You might want to check out this link
> http://www.fromdual.com/mysql-cluster-memory-sizing
>
> Regards,
> Dan.
>
> Ricardo Soares Guimarães wrote:
> > Hello,
> >
> > I have one database with 300Gb of data, and want to try NDB.
> >
> > That´s possible?
> >
> > I tried several configurations, and keys, but seems that can´t import
> > such data
> >
> > My server has (now) 16Gb of RAM, but soon will have 32.
> >
> > Anyone tried such configuration?
> >
> > I am using mysql-cluster 7.1.x with 64bits linux server.
> >
> > Thanks
> >
>
> --
> Dan FRINCU
> Systems Engineer
> CCNA, RHCE
> Streamwide Romania
>
>
> This e-mail and any files transmitted with it may be proprietary and are intended
> solely for the use of the individual or entity to whom they are addressed. If you have
> received this e-mail in error please notify the sender.
> Please note that any views or opinions presented in this e-mail are solely those of
> the author and do not necessarily represent those of ITT Corporation. The recipient should
> check this e-mail and any attachments for the presence of viruses. ITT accepts no
> liability for any damage caused by any virus transmitted by this e-mail.
>
