List:MySQL++« Previous MessageNext Message »
From:Earl Miles Date:October 26 2005 6:42pm
Subject:Re: Retrieving 300K+ records
View as plain text  
Alex Vishnev wrote:
> Chris,
> 
> The problem could very well be in iterator optimization. What I have done is
> bypassed sql++ integrator and used mysql_fetch_row directly on returned
> result set. After I obtain ResUse object reference, I retrieve MYSQL_RES
> with mysql_result() member call from ResUse object. That's how I save 1/2 of
> processing time. However, it is still not as good as loading data from a
> file. right now, it is double the time it takes to load data from the file.
> At the same time, if I go into mysql and copy all rows from one table into
> another, it takes approx 1-2s. 
> 
> mysql> create table abc select * from oldtab;
> Query OK, 397913 rows affected (1.12 sec)
> Records: 397913  Duplicates: 0  Warnings: 0
> 
> So I am thinking if internally mysql takes 1-2 sec when walking thru old
> table and creating new table, I should get similar performance. Maybe I am a
> little naïve, but.... I did not expect it to be 2 to 3 times worse than I
> have seen in db.
> 
> Alex

Well, the query you're using is probably not walking through the records, but 
using an optimization that just copies the entire file.

How long does it take to just do a "select * from oldtab" and let it scroll the 
output to the screen?
Thread
SSQLS contained within class definition compilation issueswilliam.lieberman25 Oct
  • Retrieving 300K+ recordsAlex Vishnev25 Oct
    • Re: Retrieving 300K+ recordsEarl Miles25 Oct
      • RE: Retrieving 300K+ recordsAlex Vishnev25 Oct
        • Re: Retrieving 300K+ recordsWarren Young27 Oct
      • RE: Retrieving 300K+ recordsAlex Vishnev26 Oct
        • Re: Retrieving 300K+ recordsWarren Young27 Oct
          • RE: Retrieving 300K+ recordsAlex Vishnev27 Oct
    • Re: Retrieving 300K+ recordsChris Frey26 Oct
      • Re: Retrieving 300K+ recordsEarl Miles26 Oct
        • Re: Retrieving 300K+ recordsChris Frey26 Oct
          • RE: Retrieving 300K+ recordsAlex Vishnev26 Oct
            • Re: Retrieving 300K+ recordsEarl Miles26 Oct
            • Re: Retrieving 300K+ recordsChris Frey26 Oct
  • Re: SSQLS contained within class definition compilation issuesahnkle28 Oct
    • New FeaturesFabricio Mota29 Oct
RE: SSQLS contained within class definition compilation issueswilliam.lieberman25 Oct
  • Re: SSQLS contained within class definition compilation issuesWarren Young27 Oct
Re: SSQLS contained within class definition compilation issuesWarren Young27 Oct