Hi Luke and Magnus,
2004-08-11 kl. 10.24 skrev Magnus Svensson:
>
> Hi Luke,
>
>
>> I calculate the row size as 99 bytes.
>>
>> mysql> select count(*) from gmargin;
>> +----------+
>> | count(*) |
>> +----------+
>> | 4355766 |
>> +----------+
>> 1 row in set (26.64 sec)
>>
>> and, as I understand, this sent all 4,355,766 rows over the network
>> to the mysql server, so 400,355,766 bytes, right? does this seem like
>> a reasonable time? I'm not sure how much time it is taking the mysql
>> server to count up all the rows...?
>
>
> Actually only the primary key of each row is fetched. The limiting
> factor here is that too few records are fetched at a time. We are
> working on improving the fetch algorithm too allow for more records to
> be fetched in each "batch" thus utilizing the network bandwith better.
>
As Magnus mentioned an optimisation is worked on. Currently 16 records
at a time is fetched per fragment and there are 2 fragments per node so
thus
32 primary keys are sent per batch which is
32 * //No of records
( 8 + //Assumed primary key size
4 + //Overhead per attribute
12 + //Overhead per record
20/4 //Overhead per message
) => 928 Bytes in practice 960 bytes (8 messages).
Sending 960 bytes at a time over TCP/IP doesn't give any good
performance. So we are ensuring more records sent at a time to get at
more like 32k-64k
per batch.
The calculations show that you transported 4,355,766 * 30 =
130,672,980/26,64 = 4,905,141 bytes per second => ~39Mbit/sec which is
very far from the
Gigabit. The optimisation we are doing should hopefully improve this
considerably. Going from 16 to 64 increased throughput with 27,5% for a
debug
version.
4,355,766/26,64 = 163,504 records per second.
Even with these optimisations it will still be difficult to get more
than say 300k - 500k records per second through. This is limited by
processing time.
But we'll see when the optimisations are ready what the measurements
says.
Rgrds Mikael
PS: I took a look at the web site you had found. Tampering with delays
to RxInt and TxInt can most likely provide some benefits since it
decreases the processor
usage for interrupt handling. Haven't tampered with these myself so I
have no idea what is appropriate figures.
>>
>> 400MB/26.6s = 15MB/s or 120 Mbit.
>>
>> basically, we're interested in all the settings we can change to max
>> out the performance on our cluster. is there any kind of optimization
>> guide for the cluster?
>
> In order to help you I will need some more knowledge concerning the
> kind
> of queries your application will be running, I assume you will not only
> do "select count(*)". :)
>
> MySQL Cluster is very good at primary key and unique index lookups,
> where a hash index is used to find the record. There are also ordered
> indexes available, which improves performance for a query a lot. So
> please give me some examples of queries that you intend to run, and
> I'll
> advise how to optimize them.
>
> Best regards
> Magnus
>
>
> --
> Magnus Svensson, Software Engineer
> MySQL AB, www.mysql.com
> Office: +46 709 164 491
>
>
> --
> MySQL Cluster Mailing List
> For list archives: http://lists.mysql.com/cluster
> To unsubscribe:
> http://lists.mysql.com/cluster?unsub=1
>
Mikael Ronström, Senior Software Architect
MySQL AB, www.mysql.com
Clustering:
http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html
http://www.eweek.com/article2/0,1759,1567546,00.asp
