Hi Mark,
On 10/17/08 2:43 PM, MARK CALLAGHAN wrote:
> Is the memory management model documented for MySQL? I want to know
> what APIs exist to allocate memory in the context of a user
> connection. Are there heaps for global, connection duration and
> statement duration allocation? And if there are, how are they
> accessed?
Quoting one of MySQL Internals document:
".. the memory root dedicated to execution of a statement will be
denoted as the runtime memory root of the statement. Because allocations
are done indirectly via THD::mem_root, THD::mem_root at any given moment
of time can point either to the permanent or to the runtime memory root
of the statement. Consequently, THD::mem_root and THD::free_list can be
denoted as 'currently active arena' of THD."
This comment also sheds some light:
/**
This memory root is used for two purposes:
- for conventional queries, to allocate structures stored in main_lex
during parsing, and allocate runtime data (execution plan, etc.)
during execution.
- for prepared queries, only to allocate runtime data. The parsed
tree itself is reused between executions and thus is stored elsewhere.
*/
MEM_ROOT main_mem_root;
Also, take a look in the relationship between THD, Statement and
Query_arena classes.
> I am curious because of a bug in MySQL 5.0 --
> http://bugs.mysql.com/bug.php?id=40097. A per-THD variable references
> stack-allocated memory used by a String instance.
Good catch.
> I found this, but it doesn't describe what/where MEM_ROOT instances
> exist in the context of a connection (THD).
> http://forge.mysql.com/wiki/MySQL_memory_handling_and_memory_handling_in_Falcon
>
> This states there is a connection duration memory heap -- use sql_alloc()
> http://forge.mysql.com/wiki/MysysLibraryAndAlgorithms
Not quite 'connection', it can be a runtime or permanent one.
> There appears to be a lot of allocation that does not use the
> per-connection MEM_ROOT. String (from sql/sql_string.{cc,h}) uses
> my_malloc and my_realloc to get memory. my_malloc and my_realloc don't
> use per-THD heaps like sql_alloc() and get memory directly from
> malloc(). Is there a goal to make all allocation done in the context
> of a user query allocated via the per-connection MEM_ROOT? It seems
> like a bad idea to allow memory references between objects from
> different allocation arenas -- things allocated from malloc()
> referencing things allocated from sql_alloc() and vice-versa.
I agree with the "not a good idea" part :)
> Finally, String provides 'operator new' that uses MEM_ROOT to allocate
> the String object from that MEM_ROOT. But if that String instance ever
> needs to reallocate memory for the contained string, it will get it
> from malloc() and not use the MEM_ROOT. Is there an unstated
> assumption that all uses of this won't need to allocate more memory?
The idea is that the String object is allocated from a memory root, but
the data (string of characters) can is stored in a passed buffer or in
memory allocated from malloc.
Regards,
-- Davi Arnaut
