List:Internals« Previous MessageNext Message »
From:MARK CALLAGHAN Date:October 31 2008 1:02pm
Subject:Re: memory allocation architecture in MySQL
View as plain text  
On Fri, Oct 31, 2008 at 4:49 AM, Michael Widenius <monty@stripped> wrote:
>
> Hi!
>
>>>>>> "Davi" == Davi Arnaut <Davi.Arnaut@stripped> writes:
>
> <cut>
>
>>> There appears to be a lot of allocation that does not use the
>>> per-connection MEM_ROOT. String (from sql/sql_string.{cc,h}) uses
>>> my_malloc and my_realloc to get memory. my_malloc and my_realloc don't
>>> use per-THD heaps like sql_alloc() and get memory directly from
>>> malloc(). Is there a goal to make all allocation done in the context
>>> of a user query allocated via the per-connection MEM_ROOT? It seems
>>> like a bad idea to allow memory references between objects from
>>> different allocation arenas -- things allocated from malloc()
>>> referencing things allocated from sql_alloc() and vice-versa.
>
> Davi> I agree with the "not a good idea" part :)
>
> The used logic is:
>
> - All things that is used only for the duration of a query is allocated
>  through mem_root (sql_alloc) except:
>  - Things that may grow (like strings)

Which string? There are several classes and structs used for strings.
Some have destructors, which can call free() as required if malloc()
was used. Others do not, which makes it much harder for them to safely
use malloc().

>  - Large blocks of memory used in one state of the query that can be
>    released early. These are things like sort buffers, range trees etc.
>  - Things in libraries that are outside of MySQL's control (like
>    hash tables)
>
> Anything that doesn't satisfy the above and are still using malloc
> should be moved to use sql_alloc(). Do you have any examples of
> objects that you think are wrongly using malloc ?
>
> Regards,
> Monty
>

It would help if there were a document on the developer wiki page that
stated this.


-- 
Mark Callaghan
mdcallag@stripped
Thread
memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLMARK CALLAGHAN17 Oct
  • Re: memory allocation architecture in MySQLDavi Arnaut17 Oct
    • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
      • Re: memory allocation architecture in MySQLMARK CALLAGHAN31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
        • Re: memory allocation architecture in MySQLMichael Widenius31 Oct
          • Re: memory allocation architecture in MySQLMARK CALLAGHAN30 Apr
            • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr
              • Re: memory allocation architecture in MySQLBrian Aker30 Apr
              • Re: memory allocation architecture in MySQLSergei Golubchik30 Apr
                • Re: memory allocation architecture in MySQLDavi Arnaut30 Apr