On Dec 23, 2010, at 8:12 PM, Eric Sepich wrote:
> I am returning 299,000 rows
You actually need to have the entire content of every one of these rows in RAM at the same
time?
> vector<mysqlpp::Row> v;
> queryb << "SELECT * FROM sports_schedule";
That's hugely inefficient.
First, you aren't predimensioning the vector, so it's going to reallocate itself multiple
times as MySQL++ fills it up one row at a time. A typical strategy for std::vector
implementations is to start small and double in size each time it needs more memory.
Ignoring that initial size, log2(299000) is ~18. Either say "v.reserve(300000)" or use
something that can grow efficiently, like std::list.
Second, "SELECT *" is a huge red flag. Do you really need every column? You can save RAM
and network bandwidth by selecting a subset of the columns.
Third, there is no WHERE clause. Do you really need every row in the table? Can the
server filter some subset of the rows for you? If not, can you do it client-side with
Query::store_if(), so that you can at least save some RAM, if not network bandwidth?
Finally, Row isn't the most RAM-efficient way to hold data with MySQL++. Switching to
SSQLS could help noticeably, if your data contains a lot of numeric columns. If it's
mostly text, the difference will be negligible.
