List:General Discussion« Previous MessageNext Message »
From:Thimble Smith Date:July 29 1999 9:29pm
Subject:Re: slow?
View as plain text  
At 14:21, 19990729, jice@stripped wrote:
>> At 08:54, 19990726, jice@stripped wrote:
>> >The perl script starts off parsing the text files at a acceptably fast
>> >rate.  When it gets around the 1500 address it's moving pretty slow.
>> >i let it run over the weekend and came in Monday morning to find that it
>> >was still running at a rate of 1 line inserted about ever 20 secs.
>
>just as an update... i did what you recommend.  it took about 10 secs to
>write the tab separated text file and about 30 secs to LOAD DATA INFILE.
>thanks for the advice.

Joe, I wonder what would happen if you used your original script,
but had NO indexes on your table(s).  I'm not sure if it takes longer
to update an index, the more entries you have.  If so, that could
explain the difference (if LOAD DATA INFILE creates the index all in
one step at the end).  You could also gang all of the inserts into a
single multi-insert statement (watch out for max_packet_length).  But
the LOAD DATA INFILE method is probably best anyway.

Tim
Thread
Perl and databasesMagnus Hammar26 Jul
  • slow?jice26 Jul
    • Re: slow?Thimble Smith26 Jul
      • Re: slow?jice26 Jul
      • Re: slow?jice30 Jul
        • Re: slow?Thimble Smith30 Jul
  • Re: Perl and databaseschchar27 Jul
Re: slow?Dwain Coufal26 Jul
Re: Perl and databasesMagnus Hammar28 Jul
  • Re: Perl and databasesBill Gerrard28 Jul
  • Re: Perl and databasesJani Tolonen29 Jul
Re: Perl and databasesMagnus Hammar30 Jul