List:Internals« Previous MessageNext Message »
From:Michael Widenius Date:October 31 2008 5:38pm
Subject:Re: memory allocation architecture in MySQL
View as plain text  
Hi!

>>>>> "MARK" == MARK CALLAGHAN <mdcallag@stripped> writes:

MARK> On Fri, Oct 31, 2008 at 4:49 AM, Michael Widenius <monty@stripped> wrote:
>> 
>> Hi!
>> 
>>>>>>> "Davi" == Davi Arnaut <Davi.Arnaut@stripped> writes:
>> 
>> <cut>
>> 
>>>> There appears to be a lot of allocation that does not use the
>>>> per-connection MEM_ROOT. String (from sql/sql_string.{cc,h}) uses
>>>> my_malloc and my_realloc to get memory. my_malloc and my_realloc don't
>>>> use per-THD heaps like sql_alloc() and get memory directly from
>>>> malloc(). Is there a goal to make all allocation done in the context
>>>> of a user query allocated via the per-connection MEM_ROOT? It seems
>>>> like a bad idea to allow memory references between objects from
>>>> different allocation arenas -- things allocated from malloc()
>>>> referencing things allocated from sql_alloc() and vice-versa.
>> 
Davi> I agree with the "not a good idea" part :)
>> 
>> The used logic is:
>> 
>> - All things that is used only for the duration of a query is allocated
>> through mem_root (sql_alloc) except:
>> - Things that may grow (like strings)

MARK> Which string? There are several classes and structs used for strings.
MARK> Some have destructors, which can call free() as required if malloc()
MARK> was used. Others do not, which makes it much harder for them to safely
MARK> use malloc().

I was thinking about String here.
For LEX_STRING we usually use sql_alloc() to allocate strings.

>> - Large blocks of memory used in one state of the query that can be
>> released early. These are things like sort buffers, range trees etc.
>> - Things in libraries that are outside of MySQL's control (like
>> hash tables)
>> 
>> Anything that doesn't satisfy the above and are still using malloc
>> should be moved to use sql_alloc(). Do you have any examples of
>> objects that you think are wrongly using malloc ?
>> 
>> Regards,
>> Monty
>> 

MARK> It would help if there were a document on the developer wiki page that
MARK> stated this.

Agree that this should be in the mysql-internal's wiki.

I tried to add this, but the forge didn't allow me to login :(

Will try to fix this as soon I can solve the login problem...

Regards,
Monty

Thread
memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLDavi Arnaut17 Oct
    • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
      • Re: memory allocation architecture in MySQLMARK CALLAGHAN31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
          • Re: memory allocation architecture in MySQLMARK CALLAGHAN30 Apr
            • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr
              • Re: memory allocation architecture in MySQLBrian Aker30 Apr
              • Re: memory allocation architecture in MySQLSergei Golubchik30 Apr
                • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr