List:Falcon Storage Engine« Previous MessageNext Message »
From:Olav Sandstaa Date:July 10 2009 1:08pm
Subject:Re: SRLOverflowPages log record and flushing of serial log
View as plain text  
Olav Sandstaa wrote:
> Possible solutions:
>  1. Have the Gopher flush SRLOverflowPages log records to disk? 
> (question: if we do not flush the log, why bother writing the log 
> records?) My concern here is that this will be a "performance issue".
>  2. Improve how we handle DataPage::deleteOverflowPage() during 
> recovery: by either:
>       2a: check the file size, if block is beyond EOF do not attempt 
> to read it - assume it is freed.
>       2b: if beyond EOF is reported - ignore it during recovery - and 
> assume that the block is freed.
>  3. Avoid having to read in the overflow block: Is there other reasons 
> for reading it than to check if it has a "next" overflow page? If that 
> is the case, we could change it to store the entire list of overflow 
> blocks in the data page (could also improve on reading large 
> records/blobs) - (no I am not voluntering for this right now)
> Note that alternativ 2: will not handle links to following overflow 
> blocks and that can lead to permanently lost data blocks (but that is 
> a minor issue with the current price per 4K block of disk :-) ).

I have now submitted a patch based on alternative 2b. The patch is 
available here:

This patch fixes the problem with reading an overflow page that is 
beyond beyond End-of-file during recovery by doing the following changes:

 1. IO::readPage(): instead of crashing if we read a block beyond the 
end of file we now will throw an IO error exception. I also changed the 
code so that the "fatal status" that normally would be set for IO error 
will not be set if the IO error happens during recovery.

     Alternative approach (which I also test-implemented at least for 
Unix) could be to check the size of the database file at startup and 
thus avoiding the actual "read beyond end-of-file" operation.

 2. DataPage::deleteOverflowPages(): If an IO error occures during 
recovery we now handle it by "ignoring" the failure and marking the page 
as free in Page Inventory Page.

I also had to fix an error in Cache::fetchPage(). This code did not 
behave correctly if an error was thrown during an read operation. If an 
IO exception was thrown the BDB object was not released and would stay 
"forever" in the page cache and would lead to a crash during shutdown 
due to the BDB object having a referece count not 0.

I have verified that the patch solves the problem with the repro 
scenario I described in my previous email in this thread.

Feel free to review the patch.

SRLOverflowPages log record and flushing of serial logOlav Sandstaa3 Jul
  • Re: SRLOverflowPages log record and flushing of serial logAnn W. Harrison6 Jul
    • Re: SRLOverflowPages log record and flushing of serial logOlav Sandstaa6 Jul
      • Double RecoveriesJim Starkey6 Jul
    • Re: SRLOverflowPages log record and flushing of serial logOlav Sandstaa8 Jul
      • Re: SRLOverflowPages log record and flushing of serial logOlav Sandstaa10 Jul
        • Re: SRLOverflowPages log record and flushing of serial logAnn W. Harrison10 Jul