List:Internals« Previous MessageNext Message »
From:MARK CALLAGHAN Date:October 17 2008 5:28pm
Subject:Re: memory allocation architecture in MySQL
View as plain text  
On Fri, Oct 17, 2008 at 9:43 AM, MARK CALLAGHAN <mdcallag@stripped> wrote:
> Is the memory management model documented for MySQL? I want to know
> what APIs exist to allocate memory in the context of a user
> connection. Are there heaps for global, connection duration and
> statement duration allocation? And if there are, how are they
> accessed?
>
> I am curious because of a bug in MySQL 5.0 --
> http://bugs.mysql.com/bug.php?id=40097. A per-THD variable references
> stack-allocated memory used by a String instance.
>
> I found this, but it doesn't describe what/where MEM_ROOT instances
> exist in the context of a connection (THD).
> http://forge.mysql.com/wiki/MySQL_memory_handling_and_memory_handling_in_Falcon
>
> This states there is a connection duration memory heap -- use sql_alloc()
> http://forge.mysql.com/wiki/MysysLibraryAndAlgorithms
>
> There appears to be a lot of allocation that does not use the
> per-connection MEM_ROOT. String (from sql/sql_string.{cc,h}) uses
> my_malloc and my_realloc to get memory. my_malloc and my_realloc don't
> use per-THD heaps like sql_alloc() and get memory directly from
> malloc(). Is there a goal to make all allocation done in the context
> of a user query allocated via the per-connection MEM_ROOT? It seems
> like a bad idea to allow memory references between objects from
> different allocation arenas -- things allocated from malloc()
> referencing things allocated from sql_alloc() and vice-versa.
>
> Finally, String provides 'operator new' that uses MEM_ROOT to allocate
> the String object from that MEM_ROOT. But if that String instance ever
> needs to reallocate memory for the contained string, it will get it
> from malloc() and not use the MEM_ROOT. Is there an unstated
> assumption that all uses of this won't need to allocate more memory?

Now I need to figure out how to allocate memory for the buffer used by
TABLE_LIST::query.str. TABLE_LIST::query has type LEX_STRING which is
a struct with no destructor, so I can't use malloc() as that will leak
memory. The lifetime of a TABLE_LIST instance determines which
MEM_ROOT I should use but I don't know the lifetime:
* per-connection -> use THD::mem_root
* per-table -> use TABLE::mem_root
* per table share -> use TABLE_SHARE::mem_root

The per-connection heap (MEM_ROOT) is THD::mem_root and has statement
duration as dispatch_command() calls free_root() on THD::mem_root on
exit.

It is really fun to look for references to mem_root as most classes
use the same name:
* TABLE::mem_root
* TABLE_SHARE::mem_root
* Query_arena::mem_root
* st_transactions::mem_root

>
> --
> Mark Callaghan
> mdcallag@stripped
>



-- 
Mark Callaghan
mdcallag@stripped
Thread
memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLDavi Arnaut17 Oct
    • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
      • Re: memory allocation architecture in MySQLMARK CALLAGHAN31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
          • Re: memory allocation architecture in MySQLMARK CALLAGHAN30 Apr
            • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr
              • Re: memory allocation architecture in MySQLBrian Aker30 Apr
              • Re: memory allocation architecture in MySQLSergei Golubchik30 Apr
                • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr