List:General Discussion« Previous MessageNext Message »
From:Michael Widenius Date:June 29 1999 2:14am
Subject:adding data is slow
View as plain text  
>>>>> "Jon" == Jon Drukman <jsd@stripped> writes:

Jon> I've read everything I can about speeding up data insertion, but I still
Jon> can't get it to an acceptable speed.

Jon> My table is simple:

Jon> CREATE TABLE pageview (
Jon>   url char(128) DEFAULT '' NOT NULL,
Jon>   count mediumint(9) DEFAULT '0' NOT NULL,
Jon>   date date DEFAULT '0000-00-00' NOT NULL,
Jon>   site char(32) DEFAULT '' NOT NULL,
Jon>   KEY urlidx (url),
Jon>   KEY dateidx (date)
Jon> );

Jon> I'm using it store web traffic, of course.  For each day, I store every URL
Jon> requested and the number of requests.  Importing 30k rows when the table
Jon> already has somewhere around 500k rows is incredibly slow - 40 minutes!  I
Jon> load the data using the LOAD DATA INFILE command as recommended in the manual.

Jon> Dropping the indexes and recreating them after import is not a good
Jon> solution as it takes about an hour to rebuild each index.

Jon> Selects are actually quite quick - it's just importing.  Is there any way
Jon> to speed it up?

Hi!

Try increasing the size of the key_buffer variable in mysqld.

Regards,
Monty
Thread
adding data is slowJon Drukman24 Jun
  • Re: adding data is slowSasha Pachev24 Jun
    • Re: adding data is slowJon Drukman25 Jun
      • Re: adding data is slowBenjamin Pflugmann25 Jun
        • Re: adding data is slowJon Drukman25 Jun
        • Yet Another Optimization QuestionJon Drukman28 Jun
  • Re: adding data is slowDaniel Koch25 Jun
  • adding data is slowMichael Widenius29 Jun
Re: Yet Another Optimization QuestionKevin Smith29 Jun
Re: Yet Another Optimization QuestionJon Drukman29 Jun
  • Re: Yet Another Optimization QuestionBenjamin Pflugmann30 Jun
    • Re: Yet Another Optimization QuestionJon Drukman30 Jun
Re: Yet Another Optimization QuestionKevin Smith29 Jun
Re: Yet Another Optimization QuestionJon Drukman29 Jun
  • Re: Yet Another Optimization QuestionT├Ánu Samuel29 Jun
    • Re: Yet Another Optimization QuestionJon Drukman29 Jun
      • Re: Yet Another Optimization QuestionMichael Widenius30 Jun
        • Re: Yet Another Optimization QuestionJon Drukman30 Jun
RE: Yet Another Optimization QuestionNigel Parker30 Jun
  • RE: Yet Another Optimization QuestionMichael Widenius30 Jun
Re: Yet Another Optimization Question(David Sklar)1 Jul
RE: Yet Another Optimization Questiontony2 Jul
  • RE: Yet Another Optimization QuestionMichael Widenius2 Jul
  • slow joinsJon Drukman7 Oct