On Mon, 26 Jul 1999, Thimble Smith wrote:
> At 08:54, 19990726, jice@stripped wrote:
> >I recently installed mysql on a solaris 7/sparc 5 to host a database of
> >address. Using perl, DBD::mysql to parse text files containing 30,000+
> >address and then insert into the database. It's all pretty straight
> >forward. But, there is something that seems to me really strange.
> >The perl script starts off parsing the text files at a acceptably fast
> >rate. When it gets around the 1500 address it's moving pretty slow.
> >i let it run over the weekend and came in Monday morning to find that it
> >was still running at a rate of 1 line inserted about ever 20 secs. I was
> >wondering if this is a problem with this port of mysql or perl or how i
> >code. I've also done something similar to what i described but ran mysql
> >on a Linux machine and inserted 2.5 million lines that needed a much more
> >complicated parsing script, and i had much faster results. Any incite or
> >suggestions would be greatly appreciated.
>
> Joe, are you running out of memory? One thing you could try is to have
> your perl script write out a data file (tab separated data) instead of
> do the inserts. Then you could load that file with LOAD DATA INFILE in
> MySQL. See which one is slow.
>
> Tim
Tim, thanks.
The memory issue is a good point, after looking into is.
ps reports mysqld is using 1.2% MEM and 90% CPU
the perl script is using 1.1 % MEM and 4% CPU
the machine has 256 Mb of memory.
I'll give the LOAD DATA INFILE idea a try.
thanks again,
joe rice
