RE: bzr commit into mysql-6.0-falcon-team branch (vvaintroub:2875) Bug#38186, Bug#39789

[Vladislav Vaintroub]

> -----Original Message-----
> From: Kevin.Lewis@stripped [mailto:Kevin.Lewis@stripped]
> Sent: Thursday, October 23, 2008 8:22 PM
> To: Vladislav Vaintroub
> Cc: commits@stripped; 'Falcon Team'
> Subject: Re: bzr commit into mysql-6.0-falcon-team branch
> (vvaintroub:2875) Bug#38186, Bug#39789
> 
> 
> 
> >>> Vlad wrote;
> >>> Ok, with the prior example
> >>>
> >>> Create tablespace A add datafile "file" engine falcon;
> >>> Drop tablespace A engine falcon;
> >>> Create tablespace B add datafile "file" engine falcon;
> >>>
> >>> Recovery phase1 sees
> >>>
> >>> SRLCreateTableSpace for "A" and marks datafile "file" as active
> >>> SRLDropTablespace for "A" and marks datafile "file" as inactive
> >>> SRLCreateTablespace for "B" and marks datafile "file" as active
> again
> >>>
> >>> The final state of "file" is active and thus
> SRLDropTableSpace::redo
> >>> does not delete "file" when processing SRLDropTablespace, because
> it
> >>> will be used by tablespace "B" after it. The "file"  is logically
> >>> empty at this stage though
> >>> - drop tablespace succeeds only on empty tablespaces
> >>> SRLDropTableSpace::redo will remove the entries for "A" from
> internal
> >>> hashtables and lists so on
> 
> >> Kevin wrote;
> >> When the SRLDropTablespace for "A" comes in, you say the file is
> >> logically empty.  So I do not understand what is keeping it from
> being
> >> completely deleted at this point.
> >> What is still pending in the serial log or memory which may refer to
>  >> this file that keeps it from being deleted?
> 
> > Vlad wrote;
> > The tablespace "B" that refers to the same file is still pending in
> memory
> > (and in serial log).
> 
> At the point where the recovery thread processes the SRLDropTablespace
> for "A", there is nothing referring to tablespace "A", right?
> 
> And there should be nothing yet referring to tablespace "B", right?
> Or do these questions need to be asked based on which pass through the
> serial log?
>                      Tablespace "A"     Tablespace "B"
> Before pass 1
> Pass 1
> Before pass 2
> Pass 2
> Before pass Redo
> Pass Redo


Ok, let me explain how it works with "A" and "B" in the previous example
I only explain pass1 and redo, because pass1 is where create is redone and
redo
is where drop is done

PASS1
command : create tablespace A datafile "file"

      TableSpaces {{id=10,name="A",Dbb={filename="file", filehandle=6}}
	RecoveryDatafiles={"file"}
     (opened tablespace "A" if not open, marked datafile "file" as alive)

command : drop tablespace  A	
      TableSpaces {{id=10,name="A",Dbb={filename="file", filehandle=6}}
	RecoveryDatafiles={NULL}

      (marked datafile "file" as dead)

command : create tablespace "B" datafile "file"
	TableSpaces {id=10,name="A",DBB={file="file", filehandle=6},
                   id=11,name="B",DBB={file="file", filehandle=7}}
	RecoveryDatafiles={"file"}
      (added new tablespace, marked "file" alive)


REDO
command : create tablespace A datafile "file"
      (no action)

command : drop tablespace  A	
      TableSpaces {id=11,name="B",Dbb={filename="file", filehandle=7}}
	RecoveryDatafiles={"file"}
      (checked if file is alive. Yes it alive. Remove tablespace A from the
list, close filehandle 6
        do not remove the file,  is used)

command : create tablespace "B" datafile "file"
	(no action)




> > Keep in mind, that all possible tablespaces are created in memory and
> on
> > disk during pass1() or in Tablespace::boostrap(). They are never
> deleted
> > until REDO phase comes. REDO does not recreate Tablespaces - why
> should it,
> > they are already present after the pass1.
> > IF they are not present during pass1 or pass2, we ultimately suck
> during in
> > recovery on IO::read().
> 
> Now I think I understand the problem with using pass 1 to track the
> incarnations of a datafile.  The final file must exist before pass 2,
> correct?  Or do you mean that all datafiles referenced need to exist
> during pass1?
> Why not recreate the tablespaces during or after pass 1?  All other
> recovery objects that can go through multiple incarnations are only
> done
> if they are in the final incarnation of a datafile.

All files are created in pass1. some removed in redo.
What do you mean by recreate? Create? Or remove/create?

> >> If there are things done by the recovery thread that need to be
> >> followed up later by the gopher threads, then tell me what those
> are.
> >>
> >> Ultimately, I want to lead this discussion into an alternate
> solution
> >> (for part of your changes).
> >
> > This would mean the current solution is not optimal. It is quite
> > simple and the changes are limited to SRLCreateTableSpace and
>  > SRLDropTableSpace only.
> >
> > If you want all operations to skip tablespace that is deleted at the
>  > end and do this in phase1,phase2 and REDO changes are necessary.
> >
> >
> >> That is, is it possible for the
> >> SRLCreateTableSpace for "A" and SRLDropTablespace for "A" to be
> >> completely skipped since that tablespace is an old incarnation.
> >> Will it be necesary to add tablespaceIDs to all other serialLog
> >> record types?  I don't think so.
> >> A tablespace must be logicaly empty before it is
> >> dropped, so the table incarnations should be already handled.
> >
> > It will be necessary to skip all operations on this tablespace
> > during all phases. The file is already accessed during phase1.
> 
> How is a datafile that is dropped during recovery accessed during pass
> 1?

Datafile is dropped during normal processing, pass1 only creates never
removes

 
> >> We track multiple incarnations of objects through the serial log
> with
> 
> You are correct that tablespaces do not go through multiple
> incarnations. Thanks.  It is the datafile that can do that.
> 
> I notice that recoveryDatafiles is not a RecoveryObjects *, it is a
> char**.  It just stores the name of the file. This is just a full list
> of file names referenced by the serial log which can be compared to the
> various tablespaces in the tablespace manager during REDO to see if a
> datafile is being used by one of them.

Right it is not  RecoveryObjects. What I track are strings, they are not
storable in RecoveryObjects 

> By re-using tablespaces, are the files ever truncated?  What if a huge
> tablespace is deleted and replaced with a small tablespace?  Is there a
> chance that the datafile will remain huge?
Yes, this is a good point. Obviously I make no attempt to truncate a huge
file
(and I do not know yet if a file can become huge during some phase)

> What is the danger in dropping and creating all datafiles during or
> right after pass1?  I'm sorry if I sound a little dense to you.  Please
> bear with me as I try to understand.

Exceptions. bumpPageIncarations are everywhere and otherwise bunch of
getDbb's , 
they want tablespace to be present. or else.

However, in reply to ann I considered use of exceptions for control flow and
ignoring
specific exception "tablespace not there". On the second thought the idea is
not that bad.
And would save us from the need to track anything.if we replay create/delete
in phase1 and 
ignore this specific exception in phase2 and in redo for every  serial log
record. Could work..