List:Internals« Previous MessageNext Message »
From:Stewart Smith Date:March 30 2011 7:14am
Subject:Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?
View as plain text  
On Tue, 29 Mar 2011 07:59:15 -0400, Zardosht Kasheff <zardosht@stripped> wrote:
> On Tue, Mar 29, 2011 at 7:58 AM, Zardosht Kasheff <zardosht@stripped> wrote:
> > Hello Dmitry and Sergei,
> >
> > I will look to see how NDB grabs the frm file. The reason I want it is
> > that i do not want to risk the frm file and table going out of sync.
> > It is not a problem now, but I am trying to be proactive.

You're mostly screwed in getting this perfect in MySQL - we still get
issues every so often with NDB. The solution there is to remove the FRM
files from disk and let table discovery fix it up.

In NDB we also have checks that notice when they're out of sync and then
try and pull the table definition down so that on the next access
everything works.

> > Internally, our storage engine is fully transactional. So, what I want
> > to do is save the new frm file for the altered table in some metadata
> > dictionary with the same transaction that performs data operations for
> > the alter (e.g. adds a column to every row of the table). This way, if
> > there is a crash, our engine can produce what the frm file should be
> > and the user can recover to a well known state. Right now, we just
> > cross our fingers and hope the frm file is in sync with the data in
> > the tables.

NDB is certainly what to look at here. It's as close as you can
reasonbly get in the MySQL code base to getting this right.

If you do a Drizzle port, it is possible to do exactly what you describe
- and you never need to store the table definitions on disk if you don't
want to - it's just an API and a data structure that your engine needs
to give the server, you could completely construct it out of your own
data dictionary, fairly easily too.

-- 
Stewart Smith
Thread
storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff29 Mar
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith29 Mar
  • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Sergei Golubchik29 Mar
    • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Dmitry Lenev29 Mar
    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar
Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff29 Mar
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
    • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Sergei Golubchik30 Mar
    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar
      • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
        • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff30 Mar
          • Re: storage engine access to serialized version of a TABLE orTABLE_SHARE?Dmitry Lenev30 Mar
            • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff6 Apr
              • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Jonas Oreland6 Apr
                • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith7 Apr
                  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Zardosht Kasheff7 Apr
                    • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith8 Apr
  • Re: storage engine access to serialized version of a TABLE or TABLE_SHARE?Stewart Smith30 Mar