List:MySQL++« Previous MessageNext Message »
From:Chris Morgan Date:May 5 2009 8:03pm
Subject:Re: Why Does SSQLS populate_* Function use Inefficient operator[]?
View as plain text  
Hi Warren -


> Could you compare against Query::storein(), passing a vector you've called
> reserve(50000) on first?  That would separate these issues.


I couldn't find a version of storein() that supported multi-queries.
However, I do pre-allocate the vector memory before starting the
construction, so I don't think the memory allocation overhead is the
problem.  We isolated the constructor (and not the vector insertion or
memory allocation that goes with it) by putting a simple timer around the
constructor call.  We then add up the times to get the ~1.5 seconds.  This
same timer is used for all of the other portions of the code.  The memory
allocation for all of the objects takes ~0.3 seconds.

We had received an optimization request from our users (this code is part of
>> a GUI).
>
>
>
> If it doesn't have a progress bar, add one.  Your users will then thank you
> for making the code run faster.  The UI appears to be doing something very
> fast, instead of sitting there locked up for several seconds.  That's
> headology, that is.
>
> If you've already taken care of that, this is a good use of threads: spin
> the query processing off into a thread simply to avoid blocking the UI loop.
>  You probably need this to make the progress bar work smoothly anyway.
>

I mis-typed earlier, this code is not directly part of a GUI, but it runs on
a server upon request from a GUI.  The results are then sent back to the
client GUI.  We have a separate request for a progress bar already, so we'll
be doing that for sure.

We think we just found a way to optimize the queries in the stored procedure
to save some more time, so we're going to leave the SSQLS as-is for now.
Thanks for taking the time to reply, it helped me understand a bit more of
the inner workings of the MySQL++ library.

-Chris

On Tue, May 5, 2009 at 11:13 AM, Warren Young <mysqlpp@stripped> wrote:

> On May 5, 2009, at 10:10 AM, Chris Morgan wrote:
>
>  the stored procedure that does the fetching [as well as some other
>> related queries] takes ~3 seconds total [plus about 2 seconds for the
>> store_next() calls] compared to the ~1.5 seconds that the SSQLS population
>> takes [for ~40000 rows out of 100's of millions]).
>>
>
> If you stop here, yes, the numbers appear rather frightening: almost 25%
> appears to be in SSQLS population.  But, I think you might be mixing memory
> allocation overhead in here, too.  Creating and then destroying SSQLS
> objects can be expensive, especially if there are strings or blobs in there.
>
> Could you compare against Query::storein(), passing a vector you've called
> reserve(50000) on first?  That would separate these issues.
>
>  We had received an optimization request from our users (this code is part
>> of
>> a GUI).
>>
>
> If it doesn't have a progress bar, add one.  Your users will then thank you
> for making the code run faster.  The UI appears to be doing something very
> fast, instead of sitting there locked up for several seconds.  That's
> headology, that is.
>
> If you've already taken care of that, this is a good use of threads: spin
> the query processing off into a thread simply to avoid blocking the UI loop.
>  You probably need this to make the progress bar work smoothly anyway.
>
>
> --
> MySQL++ Mailing List
> For list archives: http://lists.mysql.com/plusplus
> To unsubscribe:
> http://lists.mysql.com/plusplus?unsub=1
>
>

Thread
Why Does SSQLS populate_* Function use Inefficient operator[]?Chris Morgan5 May
  • Re: Why Does SSQLS populate_* Function use Inefficient operator[]?Warren Young5 May
    • Re: Why Does SSQLS populate_* Function use Inefficient operator[]?Warren Young5 May
      • Re: Why Does SSQLS populate_* Function use Inefficient operator[]?Chris Morgan5 May
        • Re: Why Does SSQLS populate_* Function use Inefficient operator[]?Warren Young5 May
          • Re: Why Does SSQLS populate_* Function use Inefficient operator[]?Chris Morgan5 May