List:Cluster« Previous MessageNext Message »
From:Johan Andersson Date:May 9 2012 8:00pm
Subject:Re: NDBCluster Load Data Infile extremely slow
View as plain text  
Hi,

* Split the load data into several files and load in parallel.
* avoid blob/text if you can
* set ndb_batch_size
* if you have auto_increments, ndb_autoincrement_prefetch_sz makes a big
difference
See more here about the above:
http://johanandersson.blogspot.se/2012/04/mysql-cluster-how-to-load-it-with-data.html

Best regards,
Johan
Severalnines AB

On Wed, May 9, 2012 at 9:47 PM, Scott Sandler <ssandler@stripped> wrote:

> I've found that the performance of LOAD DATA INFILE for an NDBCluster is
> over 130x slower than innodb. I expect it to be slower since it has to
> insert into multiple nodes and push data over the network, but the
> performance difference I'm seeing is 20,000 rows per second for innodb vs.
> 150 rows per second into ndbcluster (with the same schema/hardware/etc.).
> This slowness is quite a big road block in actually being able to migrate
> to MySQL cluster.
>
> Here's a pastebin of my config.ini<http://pastebin.com/JhrjdXKH>, and
> another of my my.cnf<http://pastebin.com/9y1mY7zm>. Are there any
> parameters I can change or anything I can try to speed up the ndb data
> loading?
>
> Thanks,
>
> Scott
>
>

Thread
NDBCluster Load Data Infile extremely slowScott Sandler9 May
  • Re: NDBCluster Load Data Infile extremely slowJohan Andersson9 May
    • RE: NDBCluster Load Data Infile extremely slowScott Sandler9 May
      • Re: NDBCluster Load Data Infile extremely slowJohan Andersson10 May