Re: [Drizzle-discuss] Removing Array custom vector allocation for semi-join subqueries

[Jim Starkey]

Michael Widenius wrote:
> Hi!
>
>   
>>>>>> "Monty" == Monty Taylor <mordred@stripped> writes:
>>>>>>             
>
> Monty> Brian Aker wrote:
>   
>>> Hi!
>>>
>>> On Feb 26, 2009, at 12:21 PM, Mats Kindahl wrote:
>>>
>>>       
>>>> recycling. In that case, any pointers into that old memory that are
> still
>>>> dangling would not be useful, since the memory is likely to be
>>>> overwritten with
>>>>         
>>> mem_root or not mem_root... I consider this to be pretty bad behavior to
>>> rely on.
>>>       
>
> Monty> Not mem_root. C++ has much better ways of dealing with this sort of
> Monty> thing (automatic delloc when you're done with stuff) mem_root was likely
> Monty> very important when it was written, and a great idea then.
>
> Monty> Now it's just extra complexity that hides behavior.
>
> For certain objects, memroot is a much better way to do it than with
> C++ deallocators:
>
> - Uses less memory overhead for objects
> - Less mutex, as you mainly do one free per query and
> - Much faster malloc;  Mallocs are done in bigger chunks and for many
>   queries you can avoid do any mallocs at all (when the base memory you
>   have is big enough)
>
> memroot is of course not useful for objects that changes size or has a
> short life time, but it wasn't designed for these.
>
>   

Monty, aside from minor issues like breaking C++ language semantics, 
this is extremely dangerous from an engineering point of view.  The 
problem is this: If an object is properly deleted, its destructor is 
called.  If it is deleted with the pool, the destructor isn't called, 
but the memory deleted anyway.  Valgrind isn't going to notice a 
problem, but the failure to call the destructor may leave memory subtly 
corrupted.  To work, it requires that the code and/or engineers track 
which objects need to be explicitly deleted.  Unreliability comes from 
this sort of problem where an engineer has to understand the entire 
system to make a successful local change.

If you're concerned about memory allocation/deallocation performance, 
use a thread based memory allocator that eliminates the mutex.  You get 
the performance you want leaving C++ semantics intact.  It will lead to 
cleaner encapsulation, faster code, and a lower induced bug rate.


-- 
Jim Starkey
President, NimbusDB, Inc.
978 526-1376