List:Cluster« Previous MessageNext Message »
From:Luke H. Crouch Date:August 9 2004 2:24pm
Subject:RE: gigabit performance...?
View as plain text  
does a regular mysqld server cache queries and results? when I came in this morning and
ran a query returning 27000 rows, it took 9 seconds on regular mysqld server the first
time, which was MUCH slower than the cluster (.62 seconds). but the second time I ran it,
it came back in .63 seconds...about on par with cluster...

what causes this difference?

thanks,
-L

> -----Original Message-----
> From: Mikael Ronström [mailto:mikael@stripped]
> Sent: Friday, August 06, 2004 2:26 PM
> To: Crouch, Luke H.
> Cc: cluster@stripped
> Subject: Re: gigabit performance...?
> 
> 
> Hi Luke,
> 
> 2004-08-06 kl. 20.53 skrev Crouch, Luke H.:
> 
> > we have upgraded our cluster to a gigabit network and it 
> doesn't seem 
> > that it has increased the performance very much.
> >
> > on cluster:
> > select count(*) from gmargin where gm_quote > 0;
> > 1 row in set (29.09 sec)
> > count(*) = 3786401
> >
> > on normal mysqld:
> > select count(*) from gmargin where gm_quote > 0;
> > 1 row in set (17.51 sec)
> > count(*) = 3798781
> >
> > this is about the same performance as we had on our 10/100 
> network...
> > the records are 100 bytes, and each of those records has to 
> be sent to 
> > the mysql server for counting, as I understand. so, 
> 378,640,100 bytes 
> > are being sent in 30 seconds, which is 12,621,336 bytes/sec. or 
> > 100,970,693 bits...100 megabit.
> >
> > Mikael suggested that the bottleneck was the network 
> connection, since 
> > all the records had to be returned to the mysql server. but 
> we are now 
> > using a gigabit connection between the 4 machines involved, 
> and we get 
> > the same speed.
> >
> 
> The bottleneck is in the network connection but not in its bandwidth 
> but in its use of processor resources. Gigabit Enet only provides a 
> solution when you have
> run out of bandwidth (which I don't think is the problem here). Giga 
> Enet does provide some improvement to processor performance 
> since there 
> is one
> interrupt per 8k instead of per 1500 bytes.
> 
> When scanning we can improve things somewhat by sending more 
> records at 
> a time. We are currently sending 16 at a time from the nodes.
> This is one optimisation we will need to do shortly, but also as I 
> mentioned pushing filters to the ndbd nodes will improve 
> performance by 
> a great
> factor in those cases. Our experiments show that it goes from 120k 
> records a second to 400-500k records per node. Unfortunately this 
> change
> is a bit complex so until it is in place we need to improve other 
> things like the one above.
> 
> We are also working with Dolphin Interconnect 
> (www.dolphinics.no) that 
> provides a socket implementation on a local network using proprietary 
> HW
> that decreases the load on the processor which provides 3-4 times 
> improvement in cases where network processing is bottleneck. 
> This will 
> be
> documented in the MySQL manual shortly.
> 
> > is there some kind of special setup for the mysqld server 
> to be able 
> > to handle gigabit connections? we're running that mysqld 
> (API node) on 
> > the same machine as the first DB node, could that have 
> something to do 
> > with it? do these speeds seem normal? or is the bottleneck perhaps 
> > something else besides the network?
> >
> 
> Nothing special needed.
> 
> > we're very excited about using cluster, but figure the performance 
> > should be on par or better than a regular mysql server or 
> we'll have 
> > to go to regular replication or something else...
> >
> 
> Performance issues are getting incresed priority now that the MySQL 
> integration works fairly good.
> It would be good to know which queries that you are going to use most 
> commonly so that we get your
> input into which optimisations to prioritise.
> 
> As an example is
> select count(*) from table
> a common one.
> 
> or
> 
> select's not using indexes.
> 
> Rgrds Mikael
> 
> > any ideas?
> >
> > thanks again,
> > -L
> >
> > -- 
> > MySQL Cluster Mailing List
> > For list archives: http://lists.mysql.com/cluster
> > To unsubscribe:    
> > http://lists.mysql.com/cluster?unsub=1
> >
> >
> >
> Mikael Ronström, Senior Software Architect
> MySQL AB, www.mysql.com
> 
> Clustering:
> http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
> 
> http://www.eweek.com/article2/0,1759,1567546,00.asp
> 
> 
> 
Thread
gigabit performance...?Luke H. Crouch6 Aug
  • Re: gigabit performance...?Mikael Ronström6 Aug
RE: gigabit performance...?Luke H. Crouch6 Aug
RE: gigabit performance...?Luke H. Crouch9 Aug
  • RE: gigabit performance...?Magnus Svensson10 Aug
RE: gigabit performance...?Luke H. Crouch10 Aug
  • RE: gigabit performance...?Magnus Svensson11 Aug
    • Re: gigabit performance...?Mikael Ronström11 Aug