Hi Luke,
2004-08-06 kl. 20.53 skrev Crouch, Luke H.:
> we have upgraded our cluster to a gigabit network and it doesn't seem
> that it has increased the performance very much.
>
> on cluster:
> select count(*) from gmargin where gm_quote > 0;
> 1 row in set (29.09 sec)
> count(*) = 3786401
>
> on normal mysqld:
> select count(*) from gmargin where gm_quote > 0;
> 1 row in set (17.51 sec)
> count(*) = 3798781
>
> this is about the same performance as we had on our 10/100 network...
> the records are 100 bytes, and each of those records has to be sent to
> the mysql server for counting, as I understand. so, 378,640,100 bytes
> are being sent in 30 seconds, which is 12,621,336 bytes/sec. or
> 100,970,693 bits...100 megabit.
>
> Mikael suggested that the bottleneck was the network connection, since
> all the records had to be returned to the mysql server. but we are now
> using a gigabit connection between the 4 machines involved, and we get
> the same speed.
>
The bottleneck is in the network connection but not in its bandwidth
but in its use of processor resources. Gigabit Enet only provides a
solution when you have
run out of bandwidth (which I don't think is the problem here). Giga
Enet does provide some improvement to processor performance since there
is one
interrupt per 8k instead of per 1500 bytes.
When scanning we can improve things somewhat by sending more records at
a time. We are currently sending 16 at a time from the nodes.
This is one optimisation we will need to do shortly, but also as I
mentioned pushing filters to the ndbd nodes will improve performance by
a great
factor in those cases. Our experiments show that it goes from 120k
records a second to 400-500k records per node. Unfortunately this
change
is a bit complex so until it is in place we need to improve other
things like the one above.
We are also working with Dolphin Interconnect (www.dolphinics.no) that
provides a socket implementation on a local network using proprietary
HW
that decreases the load on the processor which provides 3-4 times
improvement in cases where network processing is bottleneck. This will
be
documented in the MySQL manual shortly.
> is there some kind of special setup for the mysqld server to be able
> to handle gigabit connections? we're running that mysqld (API node) on
> the same machine as the first DB node, could that have something to do
> with it? do these speeds seem normal? or is the bottleneck perhaps
> something else besides the network?
>
Nothing special needed.
> we're very excited about using cluster, but figure the performance
> should be on par or better than a regular mysql server or we'll have
> to go to regular replication or something else...
>
Performance issues are getting incresed priority now that the MySQL
integration works fairly good.
It would be good to know which queries that you are going to use most
commonly so that we get your
input into which optimisations to prioritise.
As an example is
select count(*) from table
a common one.
or
select's not using indexes.
Rgrds Mikael
> any ideas?
>
> thanks again,
> -L
>
> --
> MySQL Cluster Mailing List
> For list archives: http://lists.mysql.com/cluster
> To unsubscribe:
> http://lists.mysql.com/cluster?unsub=1
>
>
>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com
Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
http://www.eweek.com/article2/0,1759,1567546,00.asp
