List:Internals« Previous MessageNext Message »
From:Zardosht Kasheff Date:March 30 2011 1:36am
Subject:Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?
View as plain text  
Once again, thank you Dmitry and Sergei for pointing me in the right direction.

I want to make sure that my understanding is correct. Looking at NDB,
it seems that they use the functions readfrm() and packfrm() to save
the frm file. readfrm() seems get the contents uncompressed, and
packfrm() compresses them.

Suppose I want to just save the raw uncompressed bytes that make up
the file. I think I can just use the function readfrm() to get the
data. And, if some crash caused the frm file and my saved version to
get out of sync, I can just generate a new frm file by saving the data
that readfrm() retrieved into a new frm file.

I will give the above a shot.

Thanks
-Zardosht

On Tue, Mar 29, 2011 at 7:59 AM, Zardosht Kasheff <zardosht@stripped> wrote:
> Last email did not properly go to internals
>
> On Tue, Mar 29, 2011 at 7:58 AM, Zardosht Kasheff <zardosht@stripped> wrote:
>> Hello Dmitry and Sergei,
>>
>> I will look to see how NDB grabs the frm file. The reason I want it is
>> that i do not want to risk the frm file and table going out of sync.
>> It is not a problem now, but I am trying to be proactive.
>>
>> Basically, this is what is going on. In our latest release, we have
>> ported over the new alter table API
>> (handler::check_if_supported_alter, handler::alter_table_phase1...)
>> from the MySQL cluster code branch to MySQL server 5.1, and now use
>> the function handler::alter_table_phase2 to perform a bunch of types
>> of alter table. This is not crash safe. I know that DDL statements are
>> not transactional, but I would like to do something better.
>> Internally, our storage engine is fully transactional. So, what I want
>> to do is save the new frm file for the altered table in some metadata
>> dictionary with the same transaction that performs data operations for
>> the alter (e.g. adds a column to every row of the table). This way, if
>> there is a crash, our engine can produce what the frm file should be
>> and the user can recover to a well known state. Right now, we just
>> cross our fingers and hope the frm file is in sync with the data in
>> the tables.
>>
>> Thanks
>> -Zardosht
>>
>> On Tue, Mar 29, 2011 at 7:46 AM, Dmitry Lenev <Dmitry.Lenev@stripped>
> wrote:
>>> Hello, Zardosht, Sergei!
>>>
>>> * Sergei Golubchik <serg@stripped> [11/03/29 10:57]:
>>>> Hi, Zardosht!
>>>>
>>>> On Mar 28, Zardosht Kasheff wrote:
>>>> > Hello all,
>>>> >
>>>> > Is there a way for the storage engine to get some sort of byte
> stream
>>>> > that represents a serialized version of a TABLE or TABLE_SHARE
> object?
>>>> > The goal is to save it locally during handler::create and then
> check
>>>> > its contents (as a sanity check) with the TABLE_SHARE passed in
> during
>>>> > handler::open.
>>>> >
>>>> > Is this possible? And if it is, is there a way to take this
> serialized
>>>> > data and print what the table looks like in user readable form?
>>>>
>>>> I'm afraid not.
>>>>
>>>> You can take the generated create table statement, like Stewart
>>>> suggested, or grab the complete frm file, like NDB does. But I think it
>>>> may be possible to construct an ALTER TABLE statement that modifies the
>>>> table definition (frm) only, and your sanity check will fail on the next
>>>> open.
>>>
>>> I think the latter approach should work (after all it works for NDB).
>>>
>>> In cases when ALTER TABLE statement modifies only table definition (i.e.
>>> only .FRM), we still call handler::ha_create_handler_files() method after
>>> new version of .FRM is installed, in order to inform storage engine about
>>> the fact that .FRM has been changed.
>>>
>>> --
>>> Dmitry Lenev, Software Developer
>>> Oracle Development SPB/MySQL, www.mysql.com
>>>
>>> Are you MySQL certified?  http://www.mysql.com/certification
>>>
>>
>
Thread
storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff29 Mar
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith29 Mar
  • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Sergei Golubchik29 Mar
    • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Dmitry Lenev29 Mar
    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar
Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff29 Mar
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
    • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Sergei Golubchik30 Mar
    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar
      • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
        • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
          • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Dmitry Lenev30 Mar
            • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff6 Apr
              • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Jonas Oreland6 Apr
                • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith7 Apr
                  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff7 Apr
                    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith8 Apr
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar