From: Olav Sandstaa Date: July 10 2009 12:52pm Subject: bzr commit into mysql-6.0-falcon-team branch (olav:2753) Bug#45301 List-Archive: http://lists.mysql.com/commits/78392 X-Bug: 45301 Message-Id: <20090710125214.076DF208@fimafeng09.norway.sun.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5887295128932524373==" --===============5887295128932524373== MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline #At file:///home/os136802/mysql/develop/repo/falcon-bug45301-test/ based on revid:john.embretsen@stripped 2753 Olav Sandstaa 2009-07-10 Fix for Bug#45301 Recovery crash due to fetched page marked free in PIP This crash occured during recovery due to at the time of the crash the main data page for a record had written to the data base file while the PIP and the newly allocatedoverflow page for the record had not been written. The corresponding SRLOverflowPages log record was likely not flushed to the serial log. This situation happened likely due to MySQL being killed during a Falcon check point. This is now handled by extending the IO::readPage() so that an exception is thrown in the cases there is reads beyond end-of-file during recovery. This exception is caught in DataPage::deleteOverflowPages. If it happens during recovery, DataPage::deleteOverflowPages will handle the exception by marking the overflow page as free in the Page Inventory Page. Note that if there are more than one overflow page for this record we might have permanent loss of a few blocks in the database. The patch also fixes an missing handling of exceptions in Cache::fetchPage() where an IO excecption would lead to a BDB having a use count that never got released and which resulted in a crash during shutdown. @ storage/falcon/Cache.cpp In Cache::fetchPage: Add missing handling of exceptions that might occur during reading of pages. If an exception was thrown the old code would not do proper clean-up and the allocated BDB would still be allocated and have a use count. This would never be released, take a buffer slot forever and lead to a crash during normal shutdown due to the use count on the BDB. The changed code will handle the exception, release the allocated BDB and rethrow the exception. @ storage/falcon/DataPage.cpp Add handling of read errors that occures when trying the access an overflow page that should be deleted during recovery. If an IO error occurs we handle this by ignoring this overflow page and just mark it as free in the Page Inventory Page. @ storage/falcon/Dbb.cpp Add a method to be used for checking if we currently is doing recovery on the DBB. @ storage/falcon/Dbb.h Add a method to be used for checking if we currently is doing recovery on the DBB. @ storage/falcon/IO.cpp Change how a read error that occurs during recovery is handled: 1. Do not set the "fatal error" status of the IO class 2. Throw an exception also if it is read beyond end-of-file (instead of crashing) modified: storage/falcon/Cache.cpp storage/falcon/DataPage.cpp storage/falcon/Dbb.cpp storage/falcon/Dbb.h storage/falcon/IO.cpp === modified file 'storage/falcon/Cache.cpp' --- a/storage/falcon/Cache.cpp 2009-04-09 19:15:59 +0000 +++ b/storage/falcon/Cache.cpp 2009-07-10 12:52:00 +0000 @@ -282,11 +282,25 @@ Bdb* Cache::fetchPage(Dbb *dbb, int32 pa #endif Priority priority(database->ioScheduler); - priority.schedule(PRIORITY_MEDIUM); - if (falcon_use_sectorcache) - sectorCache->readPage(bdb); - else - dbb->readPage(bdb); + priority.schedule(PRIORITY_MEDIUM); + + try + { + if (falcon_use_sectorcache) + sectorCache->readPage(bdb); + else + dbb->readPage(bdb); + } + catch (SQLException& exception) + { + // If the read operation throws an exception we need to + // release the alloacted buffer slot before re-throwing the + // exception + + bdb->release(REL_HISTORY); + throw; + } + priority.finished(); if(bdb->buffer->pageNumber != pageNumber) { @@ -376,7 +390,6 @@ void Cache::flush(int64 arg) syncWait.lock(NULL, Exclusive); sync.lock(Shared); - //Log::debug(%d: "Initiating flush\n", dbb->deltaTime); flushArg = arg; flushPages = 0; physicalWrites = 0; === modified file 'storage/falcon/DataPage.cpp' --- a/storage/falcon/DataPage.cpp 2009-06-22 12:23:12 +0000 +++ b/storage/falcon/DataPage.cpp 2009-07-10 12:52:00 +0000 @@ -30,6 +30,8 @@ #include "Log.h" #include "LogLock.h" #include "SerialLog.h" +#include "SQLException.h" +#include "PageInventoryPage.h" #ifdef _DEBUG #undef THIS_FILE @@ -375,7 +377,42 @@ void DataPage::deleteOverflowPages(Dbb * return; log->setOverflowPageInvalid(overflowPageNumber, dbb->tableSpaceId); } - Bdb *bdb = dbb->fetchPage (overflowPageNumber, PAGE_data_overflow, Exclusive); + + Bdb *bdb; + + // Read the overflow page. Note that during recovery the overflow page + // might not be found in the table space file due to having + // crashed during a check point. We handle this by catching the + // IO exception and marking the page as free. + + try + { + bdb = dbb->fetchPage (overflowPageNumber, PAGE_data_overflow, Exclusive); + } + catch (SQLException& exception) + { + + if (log->recovering && exception.getSqlcode() == IO_ERROR) + { + // Mark this page as available in PIP + + Log::debug("During recovery: reading overflow page %d failed with IO error: %s\n", + overflowPageNumber, exception.getText()); + Log::debug("During recovery: marking overflow page %d as free in PIP\n", + overflowPageNumber); + + PageInventoryPage::freePage(dbb, overflowPageNumber, transId); + break; + } + else + { + // If this exception occured during normal operation or if + // it is not an IO exception we re-throw it + + throw; + } + } + BDB_HISTORY(bdb); DataOverflowPage *page = (DataOverflowPage*) bdb->buffer; overflowPageNumber = page->nextPage; === modified file 'storage/falcon/Dbb.cpp' --- a/storage/falcon/Dbb.cpp 2009-05-12 22:58:23 +0000 +++ b/storage/falcon/Dbb.cpp 2009-07-10 12:52:00 +0000 @@ -978,6 +978,11 @@ bool Dbb::deleteShadow(DatabaseCopy *sha return false; } +bool Dbb::isRecovering() +{ + return serialLog && serialLog->recovering; +} + void Dbb::printPage(int pageNumber) { Bdb *bdb = fetchPage (pageNumber, PAGE_any, Shared); === modified file 'storage/falcon/Dbb.h' --- a/storage/falcon/Dbb.h 2009-04-09 17:31:22 +0000 +++ b/storage/falcon/Dbb.h 2009-07-10 12:52:00 +0000 @@ -130,6 +130,7 @@ public: void cloneFile(Database *database, const char *fileName, bool createShadow); void addShadow(DatabaseCopy* shadow); bool deleteShadow(DatabaseCopy *shadow); + bool isRecovering(); // Cache oriened functions void validateCache(void); === modified file 'storage/falcon/IO.cpp' --- a/storage/falcon/IO.cpp 2009-06-26 11:13:44 +0000 +++ b/storage/falcon/IO.cpp 2009-07-10 12:52:00 +0000 @@ -83,6 +83,7 @@ static int getLinuxVersion(); #include #include "IOx.h" #include "BDB.h" +#include "Dbb.h" #include "Hdr.h" #include "SQLError.h" #include "Sync.h" @@ -291,8 +292,16 @@ void IO::readPage(Bdb * bdb) if (length != pageSize) { - declareFatalError("IO::readPage", bdb->pageNumber, errno); - if (length == -1) + // Unless this error happens during recovery we set the status to + // "fatal" to prevent further operations on the file + + if (!bdb->dbb->isRecovering()) + declareFatalError("IO::readPage", bdb->pageNumber, errno); + + // If an IO error has occured (length == -1) or this is during + // recovery we throw an exception. + + if (length == -1 || bdb->dbb->isRecovering()) { throw SQLError(IO_ERROR, "read error on page %d of \"%s\": %s (%d)", bdb->pageNumber, (const char*) fileName, strerror (errno), errno); --===============5887295128932524373== MIME-Version: 1.0 Content-Type: text/bzr-bundle; charset="us-ascii"; name="bzr/olav@stripped" Content-Transfer-Encoding: 7bit Content-Disposition: inline # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: olav@stripped # target_branch: file:///home/os136802/mysql/develop/repo/falcon-\ # bug45301-test/ # testament_sha1: 8bb6a8db24fb6ca81e42f0fd448eb689dd5c7872 # timestamp: 2009-07-10 14:52:13 +0200 # base_revision_id: john.embretsen@stripped\ # 3cyny61n8afqa90p # # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWdf0kyYABoffgHAwe3f//3/n /qS////+YA8d2qAcm698uvin3t97U9dfFr7dh93t9d50OXZ9uabt9r2XUxVvuzvd92euEkmommU9 MSNlT9NSn4BNU/TKD1PST1MaA0jQ9GowmCSQTCNBNqk8iR6R5R6NTygA00NAAAaNAanppNBBNAU2 o02mqB6mm1AANGgNADE9JoJChU9T0mInlMnqbTNSNPUCGPU0GgQYQaMCPUEUiJkyMU2piaMmUep6 jKabU9Jo8oAAGgAAkSE0Q1PQmCmyhphE8psSeapiPSDIaGg0DSiWZoEKwCG0DGxjGwBtL3uMXi/1 ffcc7UDCbNsgwgEtwgetG/Oz+q0fzeBq8yKmZeHGvd/0FrlzEqbQxW2vk109G6erFlhEEIYu3vm+ n+UKiXD4fpPgV4mzAgI07TsOlJ7aAHr7qPKQ5ESlaqGDTHCcfBtiBG0Lu5nhTj+V2nNtOYmDgxj1 mabINHZCln7uozC+SLEEkkEykST7n9iWx8FzQ6OJeXYzynySEdNOTiOp4vEVNZtYsQ6aWm4dXRw7 x4f7uDDxt7/tKk5i58JXgVAQgearAmSOYjhy7qyDmz0LV69B7euKCWOqtQ/KpXYiLzvTyd3bEAm5 ztv9e/vVrlUN5nsbMxzw2JlOw/Y0DzApAouG1pQeJxgVG94e8nwZUIaJsNHyC0ZjldqwiQvqDotX MZW16UAcgLP/lAIXq8YT5gT2IaxMcY74nrpUpse/aaegzpZ6SRFnWKgOYYrDLTcbYEVGq2JPVjGw PeMpvnIUDKbYBYO09Tua8ENb0qB1CNB885Tf4p0JOg3eeCVwwaOzirG+yIlzgOJzwfa8Ywg9hXZD W3jfOyMWIbcZPa+OXAvhRsWs8QbWRS5qrSfGWSdBmF09WcbW1Q29q1va9C2E8AzRTc05irZBXRBI ITpZptoIdXFEhkFo8ItG72Pn7OVbH279O/yb9/HE/VPfBvwiQxsiiieaiMzYTm2JKv5asKrY+sNu DYwRy86osR2iHSyxneRM7ThEMmSk730O2LAVsi/Fo4fZpcHz5paNps6ZHYlqneQtEseA8GB1CnLg 7mLU9mceeNZFPBydoSYU5VScqvzpbaFtblEmzL0X11yp0VnW16KFLJQczBGaZ1sULcnBKaIlDQzu x9Ql3r2JpjHuFYi1H4RWSTTWl2OnZEcrAWKVZaZQrEdVWRmrIqJKlSglElaquKcu27dT6C+X4FjK OzrWDGUoUCFYFgywRApaUqwMqa6bLNxqMp8XCeZwzwjMrfdns5/fGUTESlDps91dt1N5EH4DF998 xJ/uwdKluZ5bMsmImBGCCJIZ6oJ2LRU2qbmSS111KlIPrzebh489J9cNdjbMXEuaETxoYxMS48gA YvQLR5xZi83MapOEFBZFVLbhkdkX3C5xQ2WpKNoZ0ywH42bnGiYRYhk7kxlYnJhKEiksm5S1WxgX JKY+lQMXkz5ix2rcmTPAcl2ousArLOwWKluRM7iZJlPdmEBPpYTEMTKwsFkXEiCDODmxmI+SxngT LiS3qawTv8Wq2V042F+oakMUmUNElrpDQx4Ha6pFfb32WhVOWnJTMzXq95lIU/MJYTVEayVWtSTr NZa+KqPU7xvCGXK0ooNao1BEuqO+XFhIsOcWZRlNpKasHg7p6jArloiYabTaaCog0RZ0espgeRTE tyzHIw4EtXj2gVInOJRJDnFpUuUgguOYzFh5r7HzPJ8B3PY2uFcOuQz6woTGjK49JcBhKUCB037p 85SrdiJklTq4mZlyRvPy71il0Ma6Dh5wNzkiSkdB0GAlVHqLxLSlbZZZdYQyILoIbdQ5BcJnQTDc QhAUX3Co4ilrx2y+LZQ2Uzwy49A12xkh7ICawHDjByS6Ovq4VIQ3HjMaZ7DbFOVFlNe23IcoRsbC JF3AxF69VzC8CGqph+hIiiAuUHpMiJNwqUfTaMsI1cBg3ajs9YcGjbbTwLPII/NroWrIaroxu4ZB o67OPmQ0K5Yc6NWEpdSrudOyWabFm22H2LlBL2YLH/IbDGcKbGhlCof15pn4M1DFiCF1rUJGOEL+ j09P+eH9VunBr+MoJY5M6rbh/D78orVjdix6AlvseU9Os1WiPQkUb7thrjHUiO2HW5qIMRuZzVHA hLTuS+4dc90Hqree0GxmYv1byVe55cZhh7GN7Y5X+/S4jsVkOErvnzTocLAp11wJtJsJestF6Bi7 TqJDPUekr9JQ715zOUUJe7iRD9DUrD5vV9Rkyh647UzJFIwTHYrVIsE0DSbGmeVHIiAxhRlEJqbq 8/J+048eY9tCNqcj/LQUaUsEqQIzu7CAJIOjNkkqVULirsziVfWwmjqq+/dwJs3XpY7rQLoOI3gz QaYVxg0NHtxrOT47xBkvDk74xhmY022keE5uE1nOMuKkisZamOs0aGmmmXr2dp8am/+4FvZtQzIf e6Hazbyrc7bp8u18L7nU4pgXmbsZhc+hziHcWl825ys+AlAzEDXQsFg1YNTBJr2PKwkauc/g6Fzb OTsbYoHBDtZn7mDGbYE1lF5yzX4JIe9CkEghQmCRJczZM78EpIMGvJkdZB/VFE2UJcEHUt4iaBkS 0QSHdEJEaBoDt5Cw7JDeOn2BuONBM9e83lBcbA7SMy85c6IuWsqIDJRULWFJnGPAdjXLw60qht0f afWQXIfAMZpWRe82sTSXGEznR4xbLBiKrZ8xOZkoijEvR0D9/HzyvTmTcIY16eCA4q0bUlanvGUt IDopJFip/1ZTvC2U7nLsY0MsXmhcqzB9gs+IIER0SBqH0FCQ1GlO5e4vpYyjaYNiojsNMjUVPKS7 Fo11+Hz2XGeDaNcA8nYGt7EplhR7GtE0pE4RwpDA0UVH7qGkkGRhIXlOleA2FgRYi0G2C14yMah4 sjtoToeGss9mDtrrkyVkrkainJK7dGgjWvYoqqIzrl7s53LDDPGh4krcSDyaLlksIeVPndYpQZIz aTFMd+epyNSBeEDO8CBkjx6ab5O4jck78eC54orajmCxUKJaDWQCpQngCPdiSIyLQYuNGCObemnE FzDuzGtCcZYdJiGXArF1HScajKXyBy8UcPijj4m10lgdBMbCCvvLTe6x9JP34j9Eln8ErSa6R4oE uziSHRCdDtZJXN8zSzorTPW74EIhQ300RSCUHehNoFpYJRg/wLWWaM0WRCbUvnEUK5bTo2aBk2kG alOwP1CToqTSeWMJAXOsCxoPbQgJAMOGEB0jQm0poPAwtbvjsoN7VBKkyrbG0zgUEsU5IMy3B1qw 4wVA1hyK00ae8kFFuGPbtZAwFgp2eEyd1BlYgmLUeaFalNo8R+6oBw888xgyNJYcszAsc4kuIYHB lpTtiiJvz1Hl6cM2VZwysENIau22+RZC1tNNhhhpYEGqNG8usBXEiL049HSnh7eeCMehY1q8Qne8 /qxKnd0Jo+BswgiODg9Ol9LLJyM5obeQFpkeuXIasTGGGtKBZLD31QG9edFtULYUa47kGwflBoMg OvhbkjDXB3YdsV3F1dVJLWgMi3hWSNJWYylKBmPFynfZZg1Qdr15cCOh1KnnTc4bWEhzokr6pdKP g7oTG4pyMexFn0oLiiWSPTKp8jTBgoh56IpGRtvgiqDrJwDB9JEjcdqy8SO5k1ED4sDSwng4Lusq aSWWFqKAKJTUkSyThoVT3RYm0pSdPtv9OVDr3tRW5bJmYlESFY0Gcgx8i13sDQ37pzUirsuXny3p hvpPQcyz9VANR+tRtli7VYsp3ms5Pi/MdyTRfiAWcGxQGIgmhoaQ+oX11xFlsMxZMXEDIyRwtYIu CXmey1kKdl1UtNibH44+Rw15+JrHjWXAOvXC7n1ssSC7EXvBA3yIghJ3MbkuXj0AHkQmrsFx4M9l OAvJ75D1+ryLOEx35RvLwtYdoVwME+pKOgEuccnjF97Bc73UJlgH7fY3Gh497PmTyuxpaPyoadJa QRjEgjngkMLuepnOFDPyzJTU0NVg1EMNpuIVrCBgsavFyyJsqi8hDJKZU9PVb1ebIm2Gd6B1pAXf G55wrlklnAlgJk2qYaLrUpAS5hGtWFbRfBkpMghh0qpQoNoGxqaclZk2aUsnMRVBz0QpdCXY9yon dE3iJzDkP1y/BYgzFiB+r3lL6xw94ZSxkD+nQkDWXgfLORdxcrQlI1jKuKnMXAyopSxeLQWpYS7k 5XjcckkkklNS3Ll28tNLAymWcY7zieCxo2oYFugSJVa1i3sbcijMVN+rhThqVGUsSYieiJLq022F CiLBEkpjwGoaY1QHmLiyZSi2So/iYNwqIj02NHTQ0RJHiYRUabvqHPWdFgZwWIWQwQrEd9DoKRea YhE1nB2kpFedkt4bk/IOnRGQY51X4lXh54KxQRFapj4ssmD9fSsU6JqK96KogSKBxRVFUL1DzSC6 DWuZnCY85gr2gPPW7ZnRs6VhqF+hhQt2c9JWQDrZcZM6gDa90k9kIYiYJAyyEuZlMJh1euU73c2J j5BdbiSPCNJsexWwXCrAGYN7ekcbhJ1ClAI9DlZEyD2xkY5NDg73a2lPocar2xp4J9LuMR209raJ YOtkOiWxbg3GT7ZYlEzRfkXckU4UJDX9JMmA --===============5887295128932524373==--