List:Cluster« Previous MessageNext Message »
From:Tomas Ulin Date:December 3 2004 9:17am
Subject:Re: Testing NDB
View as plain text  
Dathan,

There are a few things (given that we don't change hardware and 
introduce a low latency interconnect):

in 4.1.7 and earlier:
1. use fewer ndbd nodes (maybe not what you wan't, but it will be faster)
2. use more mysql servers (even on the same machine as the other mysql 
server)
3. try having one mysql server co-located on each server running an ndbd
4. in the case below you will also get better performance if you have 
NoOfReplicas=4

in 4.1.8:
(same as for 4.1.7 +)
5. set ndb_use_exact_count=off;
6. experiment with ndb_use_force_send=off;

in 5.0 (not pushed yet though, pending review):
(same as for 4.1.8 +)
7. you will have access to batching, i.e. possibility of fetching 
several "id's" at once, i.e.  select * from t where id in (1,2,3,4);

T

Dathan Pattishall wrote:

>Summary:
> 
>Taking the exact same database with tables replicating from a master in
>near real time, one in a myISAM config in pure memory the other a series
>of NDB nodes, I've found that my MYISAM version is 5 times faster then
>my NDB counter part on identical AMD Opertons 2.4 GHZ in 64bit mode with
>8 GB of ram each-running Suse Enterprise Linux 8. All nodes are
>connected via a 1 Gb link on a dedicated network to mySQL traffic. 1
>server is 5 times faster then 4 NDB storage nodes and a mySQL NDB client
>API. Is there a problem with my config? Can I tweak out the setup to get
>faster R(t) from NDB to close the gap between it and my MYSQL config?
> 
> 
> 
>Goal:
> 
>Determine best configuration for supplied hardware to achieve remarkable
>speed using NDB.
> 
>Type of Query used:
> 
>O(1) - hash lookup
> 
>select col1,col2 from table sub where id = INT 
> 
>5 Simulatenous threads all running 10K took 350 seconds while the myISAM
>counter part took 94 seconds.
> 
> 
>Process:
> 
>dathan@ndb1:~> /usr/local/mySQL/bin/ndb_mgm 
>-- NDB Cluster -- Management Client --
>Connecting to Management Server: localhost:2200
>NDB> show
>Cluster Configuration
>---------------------
>[ndbd(NDB)]     4 node(s)
>id=2    @10.16.92.101  (Version: 3.5.3, Nodegroup: 0, Master)
>id=3    @10.16.92.102  (Version: 3.5.3, Nodegroup: 0)
>id=4    @10.16.92.103  (Version: 3.5.3, Nodegroup: 1)
>id=5    @10.16.92.104  (Version: 3.5.3, Nodegroup: 1)
> 
>[ndb_mgmd(MGM)] 1 node(s)
>id=1    @10.16.66.48  (Version: 3.5.3)
> 
>[mysqld(API)]   2 node(s)
>id=6    @10.16.92.105  (Version: 3.5.3)
>id=7    @10.16.92.106  (Version: 3.5.3)
>
>NDB_MGMD management node /etc/config.ini
> 
>#
># config.ini - 1 ndb mgmd node 4 storage nodes and 2 mysqld server node
>#
>#
>#
> 
>
>[NDBD DEFAULT]
># Number of replicas for each table stored in the cluster
># Also specifies the size of node groups
># NoOfReplicas is not preportional to storage nodes and has a max of 4.
> 
>NoOfReplicas = 2
> 
>DataDir= /var/lib/mysql/mysql-cluster
># Applications that perform large transaction involving many records-
># need to have MaxNoOfConcurrentOperations increased.
># Every transaction that changes data in the cluster it needs to have-
># an operational record-it contains state information needed for-
># find UNDO records.
># SET value to:
># num of records / num of nodes if an application affects that num of
>row
>s
>MaxNoOfConcurrentOperations=30000
>DataMemory=3024M
>IndexMemory=3024M
># How often heartbeats are sent and how often to expect to recieve them
># If 3 are lost in a row the node is declared dead. 4*value in ms it's
>de
>ad
>HeartbeatIntervalDbDb=3000
># Checks the main threads for loops.
>TimeBetweenWatchDogCheck=3000
># How long the transaction coordinator will wait until it aborts the
>tran
>saction
>TransactionDeadlockDetectionTimeout=3000
> 
>MaxNoOfTables=132
> 
>[COMPUTER]
>Id=1
>Hostname=ndb1
>[COMPUTER]
> 
>Id=2
>Hostname=dbshard1
>[COMPUTER]
>Id=3
>Hostname=dbshard2
>[COMPUTER]
>Id=4
>Hostname=dbshard3
>[COMPUTER]
>Id=5
>Hostname=dbshard4
> 
>[NDB_MGMD]
># identity of this node
>Id=1
>PortNumber=2200
> 
>ExecuteOnComputer=1
> 
># 0 it isn't used as arbitrator
># 1 high priority
># 2 low priority
>ArbitrationRank=1
> 
>[NDBD]
>Id=2
>ExecuteOnComputer=2
>
>[NDBD]
>Id=3
>ExecuteOnComputer=3
> 
>[NDBD]
>Id=4
>ExecuteOnComputer=4
> 
>
>[NDBD]
>Id=5
>ExecuteOnComputer=5
> 
>
>[MYSQLD]
>[MYSQLD]
> 
> 
> 
> 
>
>DVP
>
>----
>
>Dathan Vance Pattishall     http://www.friendster.com
>
> 
>
> 
>
>  
>

Thread
Testing NDBDathan Pattishall2 Dec
  • Re: Testing NDBTomas Ulin3 Dec