List:General Discussion« Previous MessageNext Message »
From:Mark Matthews Date:December 15 2010 9:50pm
Subject:Re: Bulk Insertion Performance
View as plain text  
On Dec 14, 2010, at 7:21 PM, Feris Thia wrote:

> Hi Mark,
> 
> On Wed, Dec 15, 2010 at 8:10 AM, Mark Matthews <mark.matthews@stripped>
> wrote:
> Feris,
> 
> *How* are you writing, via batch statements with rewriting, or directly, or via LOAD
> DATA INFILE? It seems you're off by about a factor of 10-20x from what I've seen
> performance-wise for writes.
> 
> I'm using ETL mean - for this case, it is a java application name Kettle (Pentaho
> Data Integration). And it use JDBC connection. 
> 
> Is it a JDBC driver configuration ?

Feris,

I don't know what Kettle is doing under the hood, but if it's doing addBatch(),
executeBatch(), then adding "rewriteBatchedStatements=true" to your MySQL JDBC URL should
probably help quite a bit.

	-Mark
-- 
Mark Matthews
Principal Software Developer -  MySQL Enterprise Tools
Oracle
http://www.mysql.com/products/enterprise/monitor.html







Thread
Bulk Insertion PerformanceFeris Thia15 Dec
  • Re: Bulk Insertion PerformanceMark Matthews15 Dec
    • Re: Bulk Insertion PerformanceFeris Thia15 Dec
      • Re: Bulk Insertion PerformanceMark Matthews15 Dec
        • Re: Bulk Insertion PerformanceFeris Thia16 Dec
          • Re: Bulk Insertion PerformanceJohan De Meersman16 Dec
            • Re: Bulk Insertion PerformanceFeris Thia19 Dec