From: Warren Young
Date: May 5 2009 6:13pm
Subject: Re: Why Does SSQLS populate_* Function use Inefficient operator[]?
List-Archive: http://lists.mysql.com/plusplus/8549
Message-Id: <7746B78B-43B1-4B20-A073-7610192D3E20@etr-usa.com>
MIME-Version: 1.0 (Apple Message framework v930.3)
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

On May 5, 2009, at 10:10 AM, Chris Morgan wrote:

> the stored procedure that does the fetching [as well as some other
> related queries] takes ~3 seconds total [plus about 2 seconds for the
> store_next() calls] compared to the ~1.5 seconds that the SSQLS  
> population
> takes [for ~40000 rows out of 100's of millions]).

If you stop here, yes, the numbers appear rather frightening: almost  
25% appears to be in SSQLS population.  But, I think you might be  
mixing memory allocation overhead in here, too.  Creating and then  
destroying SSQLS objects can be expensive, especially if there are  
strings or blobs in there.

Could you compare against Query::storein(), passing a vector you've  
called reserve(50000) on first?  That would separate these issues.

> We had received an optimization request from our users (this code is  
> part of
> a GUI).

If it doesn't have a progress bar, add one.  Your users will then  
thank you for making the code run faster.  The UI appears to be doing  
something very fast, instead of sitting there locked up for several  
seconds.  That's headology, that is.

If you've already taken care of that, this is a good use of threads:  
spin the query processing off into a thread simply to avoid blocking  
the UI loop.  You probably need this to make the progress bar work  
smoothly anyway.