Hi!
On Dec 14, Hagen H?pfner wrote:
> Dear List,
>
> i am back from a business trip, where I had a lot of time to try to
> understand the remainer of the MYI-Structure. Unfortunately, the
> internals documentation and the comments in the source code raise more
> question than they answer ;-( However, let me again try to describe,
> what I believe and say yes if I am right in point or correct my answer,
> please ... The structure was taken from mi_open.c:
> ------------------------------------------------------------
> mi_sizestore(ptr,base->keystart); ptr +=8;
> I assume that this stores again the start of the key values of the
> first key ...
Yes.
Well, technically, not "of the first key" because when you add and
delete values to the indexes new keypages are allocated, and free'd,
moved to "deleted" list and taken from there, so you may end up having
the first keypage in the file used by some other index, not the first
one.
> ------------------------------------------------------------
> mi_sizestore(ptr,base->max_data_file_length); ptr +=8;
> mi_sizestore(ptr,base->max_key_file_length); ptr +=8;
> Both values are corrected in mi_open.c but why are they stored in the
> MYI-file if they are corrected with each open?
> I assume, that they describe the maximal possible length of the MYD file
> or the MYI file respectively. But why and when it is really necessary to
> correct them.
No idea :)
Perhaps it has something to do with UNIREG
> ------------------------------------------------------------
> mi_rowstore(ptr,base->records); ptr +=8;
> mi_rowstore(ptr,base->reloc); ptr +=8;
> I have got no idea what this values encode/mean?
> ------------------------------------------------------------
> mi_int4store(ptr,base->mean_row_length); ptr +=4;
> I believe that the name says what the meaning is, but I was not able to
> find any function that uses this value and in my tests the value stayed
> 00 00 00 00 ... so is it really in use?
Apparently not.
> ------------------------------------------------------------
> mi_int4store(ptr,base->reclength); ptr +=4;
> This seams to be the lenght of noncompressed/unpacked records (described
> in myisamdef.h) but how is it computed? In the internals doc I found the
> information, that reclength is the sum of the lengths of all attributes
> of the relation. But the example focuses on fixed length columns only. I
> tried to calculate reclength for a table with an int, a varchar(100) and
> a text attribute. The reclength in the MYI-file is 116 (0x00 00 00 74)
> ... is 116 the sum of 4 (int) and the maximal "real" lengths of the both
> other attributs?
0 (nulls) + 4 (int) + 100 (varchar) + 8+4 (text)
that means - all you columns were defined NOT NULL
int takes 4 bytes
varchar takes 100 bytes (not on the disk, but in the row image in
memory)
text takes (in the row image in memory) 4 bytes to store the length and
sizeof(char *) to store the pointer to the blob data.
> ------------------------------------------------------------
> mi_int4store(ptr,base->pack_reclength); ptr +=4;
> Is this the average for a packed record? If so, how is it computed? If I
> do not pack the data file, then pack_reclength is equal to reclength,
> isn't it?
How it's computed - see mi_create.c.
It's equal if there's no live checksum, and pack_reclength
is not less than rec_reflength.
Not the average, it's basically the reclength above without
rec_reflength/checksum correction.
> In mi_static.c i found the following:
> if (info->state->data_file_length >
> info->s->base.max_data_file_length-
> info->s->base.pack_reclength)
> ... means data file is full
> Does this mean, that I got the error, if it would not be possible to
> store a new packed record?
unless you delete some rows, yes.
You can rebuild the table to increase max_data_file_length though.
> ------------------------------------------------------------
> mi_int4store(ptr,base->min_pack_length); ptr +=4;
> mi_int4store(ptr,base->max_pack_length); ptr +=4;
> This are the minimal and the maximal possible legth of packed records,
> aren't they?
Yes.
> ------------------------------------------------------------
> mi_int4store(ptr,base->min_block_length); ptr +=4;
> Minimal block length of what? Of an index page?
No, it's minimal length of the "chunk" in MYD of dynamic format, see
mi_dynrec.c
> ------------------------------------------------------------
> mi_int4store(ptr,base->fields); ptr +=4;
> Number of attributes +1 (so it is described in the internals doc). But
> why do you need the +1?
Not always +1, I believe.
It's for a internally created field (MyISAM specifics, other storage
engines don't see it) to have NULL bitmap in the row image to belong to
some field. The field is created in ha_myisam::create() :
if (recpos != minpos)
{ // Reserved space (Null bits?)
bzero((char*) recinfo_pos,sizeof(*recinfo_pos));
recinfo_pos->type=(int) FIELD_NORMAL;
recinfo_pos++->length= (uint16) (minpos-recpos);
}
> ------------------------------------------------------------
> mi_int4store(ptr,base->pack_fields); ptr +=4;
> What does this mean. In my example, described above, this value is 1 but
> I do not have a packed field?
What's the table definition ?
If the one with varchar(100) and text - then you have two packed fields.
(see how "packed" is incremented in mi_create.c)
> ------------------------------------------------------------
> *ptr++=base->rec_reflength;
> *ptr++=base->key_reflength;
> I assume this is the length of the pointers either refering another
> key-entry or a record in the data-file. Both values are 4 in my example.
Yes.
> ------------------------------------------------------------
> *ptr++=base->keys;
> The number of keys.
yes.
> ------------------------------------------------------------
> *ptr++=base->auto_key;
> Which key is indexes the auto increment attribute. Zero if no auto
> increment attribute.
yes.
> ------------------------------------------------------------
> mi_int2store(ptr,base->pack_bits); ptr +=2;
> I have no idea ;-)
I don't know either. In mi_create.c it's set as
share.base.pack_bits=packed;
share.base.pack_fields=packed;
so it's identical to pack_fields.
Perhaps it was different long time ago, but not anymore...
> ------------------------------------------------------------
> mi_int2store(ptr,base->blobs); ptr +=2;
> Number of blobs. But why it is 1 in my example. I do not have a blob or
> a text attribute going to be interpreted as a blob?
Yes, TEXT is a blob.
The only difference is the character set.
> ------------------------------------------------------------
> mi_int2store(ptr,base->max_key_block_length); ptr +=2;
> maximal allowed size of a key page ... predifined 1024 ... but in
> myisam.h MI_MAX_KEY_BLOCK_LENGTH is defined as 16384. What is the
> difference between MI_... and max_key_...?
MI_MAX_KEY_BLOCK_LENGTH is the max. supported by MyISAM,
max_key_block_length is the max. used in this MYI file.
> ------------------------------------------------------------
> mi_int2store(ptr,base->max_key_length); ptr +=2;
> Maximal allowed length of a key value (includes the pointer) ... In my
> example (with a fulltext index) the value is 272 (0x0110). Where and how
> is is calculated?
in mi_create.c
it's 254 (max. length of the word) + 4 (float) + 1 (to store the length
of the word 1..254), that is 259. And finally
#define ALIGN_SIZE(A) MY_ALIGN((A),sizeof(double))
share.base.max_key_length=ALIGN_SIZE(max_key_length+4);
which gives you 272 (259+5 aligned to 16-byte boundary)
> ------------------------------------------------------------
> mi_int2store(ptr,base->extra_alloc_bytes); ptr +=2;
> *ptr++= base->extra_alloc_procent;
> I do not have any idea ;-)
As you can see, it's not used anywhere.
> ------------------------------------------------------------
> *ptr++= base->raid_type;
> A flag that indicates the used RAID type ... It was specified during
> CREATE TABLE ... From the mysql doc i learned, that i have three
> options:{ 1 | STRIPED | RAID0 }. Unfortunately I was not able how these
> three (or two, I typically mix up RAID 0 and RAID 1 but one mens
> STRIPED, so what is the difference here) types are encoded and
> represented in this one byte in the MYI file.
Forget it. RAID support is removed starting from 5.0.
> ------------------------------------------------------------
> mi_int2store(ptr,base->raid_chunks); ptr +=2;
> mi_int4store(ptr,base->raid_chunksize); ptr +=4;
> Both values come from the create table statement?
Yep.
> ------------------------------------------------------------
> bzero(ptr,6); ptr +=6; /* extra */
> Hm, I have absolutely no idea, why you researved 6 extra bytes, here ...
Alignment, I believe.
> Are they used anyhow?
No.
> ------------------------------------------------------------
>
> Again, questions over questions, but I seriously try to understand the
> MyISAM internals ;-)
I can see that :)
Regards,
Sergei
--
__ ___ ___ ____ __
/ |/ /_ __/ __/ __ \/ / Sergei Golubchik <serg@stripped>
/ /|_/ / // /\ \/ /_/ / /__ MySQL AB, Senior Software Developer
/_/ /_/\_, /___/\___\_\___/ Kerpen, Germany
<___/ www.mysql.com
