Re: BLOB implementation for heap engine

[Ingo Strüwing]

Hi Emanuele,

ciccettino@stripped schrieb:
> Hello,
> Firstly I want to thank you all for the fast answers and the support.
> I've somo other questions: 
> 
> 0) What is the structure of a heap table in memory? How infos about columns are
> stored and how rows are memorized? And when an existent 
> table is opened, from where all its infos are retreived?

HP_INFO (include/heap.h) is the central structure of a heap table. It is 
initialized when the table is opened.

> 1) I found that in the creation of a heap table it seems that just infos about keys
> are stored: in fact in the HP_CREATE_INFO struct no 
> info about other fields are utilized. The function heap_create() in hp_create.* takes
> as parameters this HP_CREATE_INFO and just infos 
> about keys and record length. Where are things about non-key fields and how are
> managed? And where the fixed length of rows for a table is 
> setted?

Heap tables store fixed length records. There is no need (yet) to know 
anything about the fields (columns). They do not need to be compressed 
or otherwise treated specially (yet).

Only the keys need to know where in the records the relevant information 
is located. But that's stored in the keyinfo.

The fixed record length is stored in HP_INFO::s->reclength.

> 2) If I don't misunderstand, in the function heap_create() the memory blocks for the
> table rows are initialized with the instruction 
> "init_block(&share->block, reclength + 1, min_records, max_records)", in which
> "reclength" comes from the TABLE_SHARE (SQL layer): now this 
> parameter, when in a table there are blobs, computes the record length where the
> length of blob field is sizeof(pack_length)+sizeof(char*) 
> or sizeof(pack_length)+pack_length? Because in the first case, reclength would be the
> same for my heap tables with blobs, but in the second 
> one, I may calculate a new_reclength where I shoul subtract the pack_length for all
> the blobs and add something like num_of_blobs*sizeof
> (char*)...

I believe it is the first one (length+pointer). But you need to verify 
it by experimenting.

> 3) I analyzed mi_dynrec.c and the suggested function _mi_rec_pack(): thanks to this,
> I understood how blob fields are managed in isam 
> storage engine. A thing that I'd have explained is the real meaning of the word/verb
> pack in this context, which seems to be linked with 
> dynamic rows. What does it mean the hermetic comment /* Pack a record. Return new
> reclength */. I interpredet it as: copy in record[0] the 
> pack_length and the pack itself, but I'm not sure.

The verb 'pack' means something like 'compress'. The records are taken 
column by column from 'from' (record[0]) and put (packed) most 
efficiently into the buffer 'to'. This is often a temporary buffer or 
MI_INFO::buff. In case of a blob column this means to copy the blob data 
from the referenced memory buffer to the 'to' buffer at its place 
between the other columns. The result can then be used to write into the 
data file.

> I'm trying to start modifications, but I want to be sure that I don't make stupid
> mistakes...

I see. But we all can't completely avoid that. ;-)

Regards
Ingo
-- 
Ingo Strüwing, Senior Software Developer
MySQL GmbH, Radlkoferstr. 2, D-81373 München
Geschäftsführer: Hans von Bell, Kaj Arnö - HRB München 162140