Hi.
Am Mittwoch, den 28.02.2007, 22:40 -0800 schrieb Mark Callaghan:
> On 2/28/07, ciccettino@stripped <ciccettino@stripped> wrote:
...
> > I'm getting mad with some aspects that aren't so clear just reading the
> > source code and the documentation available (to me, MySQL
> > Internals documentation could and should be more comprehensive, deepening
> > many other aspects and details;
...
> Except for your desire for UML, I know how you feel.
I fully agree. Documenting is usually not the favorite pursuit of
developers. :(
I have started to try to add comments to code that I need to understand
and/or write small chapters that are intended for the internals manual.
Unfortunately in most situations the time is missing to complete the
chapters. So I was not too successful with this by now. :(
So there might possibly be some text fragments hanging around with a
couple of devs. But with respect to the memory storage engine I doubt
so. There is definitely no "tangible"/"internal" documentation besides
our internals manual.
> > First of all, in a table struct what are "fields" and what are "records"?
...
> > "items"? Which of these structures stores a row?
>
> 'fields' corresponds to the columns in a row, and some methods and variables
> from it can be used to determine the location within record[0] for a column.
Agree. "record" is almost the same term as "row", btw.
However, the storage engine does not have access to the 'field'
structures. It cannot use their pointers to locate every column. It has
its own structures which describe the layout of a record of this table.
For example, MyISAM has this (myisamdef.h):
MI_COLUMNDEF *rec; /* Pointer to field information */
Unfortunately I found nothing similar for memory tables. This could be
the case because it uses fixed length records and thus doesn't need to
know what is where within the record. I think this would be required
when implementing blobs and must be filled out in heap_create().
Therfore the column definitions must be extracted from the table object
in ha_heap::create() and supplied to heap_create().
I recommend to compare with MyISAM code. This has the most similarities
with MEMORY compared to other storage engines.
> record[0] is the buffer used to pass the row between storage engines and the
> SQL layer
>
> I am not sure what record[i] for i > 0 does.
record[1] is used in some cases where temporary record storage is
required or old and new records are needed. For an update the storage
engine is handed over the new and old record. So it can locate the old
record and replace changed columns and the keys belonging to these
columns. On table open it is decided if the second record storage is
required for this table (see sql/table.cc:open_table_from_share()
(5.1)).
...
> > ha_write_row(table->record[0]), which seems to just write out a
> > buffer (the "table->record[0]" that I didn't understand what precisely is)
As mentioned above this is the record (in an in-memory presentation)
that is to be inserted in the table. The storage engine knows where the
columns are located that need some tweaking to construct an in-file
representation of the record and locate the columns that contain
key(part)s for the indexes. For example varchar columns are condensed to
their used length. Blobs have pointers, which point to elsewhere in
memory and need to be copied from there into the in-file representation.
> > in a custom way that depends on the storage engine. Now my main
> > question is: if, as said, a blob is stored in memory as a lenght and a
> > pointer (in a blob field there is the "String value" which stores
> > the real data, isn't it?), how and where the pointed data is inserted in
> > the buffer that then the handler writes out (when, of course, the
> > storage engine supports blobs)?
I hope my above explanation was clear enough. But I'll make another
attempt. Blob data is allocated with something similar to malloc(). So
it gets some random memory address. This address is placed in the record
buffer (table->record[0]) at the place where the column data are
expected. Hence the storage engines needs to have the record buffer
only. It finds the blob data address in it and can copy the data from
that address. Note that the whole MySQL server is one process and shares
the whole address space. Even if storage engine and SQL layer would be
different threads, they would share that blob data.
> For SELECT statements, when the storage engine returns a row with a LOB
> column, a pointer is returned in record[0] that points to the LOB and the
> memory that is pointed to is owned and managed by the storage engine
> instance.
I'm just not sure if it is the storage engine that frees()s the (B)LOB
data again or if it's done by the receiving SQL layer in this case. But
this should not be too difficult to find out.
Regards
Ingo
--
Ingo Strüwing, Senior Software Developer
MySQL GmbH, Radlkoferstr. 2, D-81373 München
Geschäftsführer: Hans von Bell, Kaj Arnö - HRB München 162140
