From: Kevin Lewis Date: July 14 2009 2:10pm Subject: bzr commit into mysql-6.0-falcon-team branch (kevin.lewis:2755) Bug#45746 List-Archive: http://lists.mysql.com/commits/78652 X-Bug: 45746 Message-Id: <0KMR00HR8ZD2S210@mail-amer.sun.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="Boundary_(ID_CHGjJBmMvq0bo/b6oEnU1g)" --Boundary_(ID_CHGjJBmMvq0bo/b6oEnU1g) MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII Content-transfer-encoding: 7BIT Content-disposition: inline #At file:///C:/Work/bzr/Merge/mysql-6.0-falcon-team/ based on revid:kevin.lewis@stripped 2755 Kevin Lewis 2009-07-14 Bug#45746 - Gopher thread gets error 305. The previous patch for this bug made it highly unlikely for the gopher thread to get error 305-out-of-record-cache. But it is still possible since thawAll() still may need to muck with the record cache. So this patch makes the gopher thread retry the attempt to allocate a record another 10 time. Client threads which also get this error will not retry so they will probably rollback and free some memory. In thawAll(), the gopher will also give up if the transaction gets too old while this is happening. Then after 10 tries, which should take more than 22.5 seconds, The gopher will just give up trying to thaw that record, which may never be needed anyway. If it is needed, the client thread must be able to recognize an unthawable record. If thawAll does not thaw a record, it will remain thawable from the serial log until the serial log toggles so that the virtual offset is no longer around. So this patch uses two new indicators to identify an unthawable record. The first is a new state called recBuried which indicates that the record version has been replaced in the page. This is now set in thawAll(), during transaction completion. Then later, if a 'buried' record cannot be thawed from the serial log, RecordVersion::thaw() does not attempt to thaw it from the page. It just marks it as unthawable by setting Record::data.record = recordDataIsFreed. Then the client thread in fetchVersion() must throw an error indicating that a visible record cannot be thawed. @ storage/falcon/Record.cpp Bug#45746 - account for new state recBuried @ storage/falcon/Record.h Bug#45746 - new state recBuried @ storage/falcon/RecordVersion.cpp Bug#45746 - fetchVersion() should recognize an unthawable record It throws an internal error with an appropriate message. RecordVersion::thaw() now avoids thawing a record from the data page that has been superceded. @ storage/falcon/Table.cpp code cleanup @ storage/falcon/Transaction.cpp Bug#45746 - thawAll() is enhanced to attempt to thaw arecord 10 time if the error is 305-out-of-record-memory. It will also quit early if the transaction becomes too old. Then if 10 loops occur, it will just ignore that record, hoping that it is not actually needed before the serial log toggles and it becomes unthawable. thawAll also now marks prior records that have been superceded by these completed records as recBuried. the data page, or 'buried'. modified: storage/falcon/Record.cpp storage/falcon/Record.h storage/falcon/RecordVersion.cpp storage/falcon/Table.cpp storage/falcon/Transaction.cpp === modified file 'storage/falcon/Record.cpp' --- a/storage/falcon/Record.cpp 2009-07-08 20:05:22 +0000 +++ b/storage/falcon/Record.cpp 2009-07-14 14:09:58 +0000 @@ -1094,7 +1094,7 @@ char* Record::allocRecordData(int length // Give the scavenger thread a chance to release memory. // Increase the wait time per iteration. - Thread *thread = Thread::getThread("Database::ticker"); + Thread *thread = Thread::getThread("Record::allocRecordData"); thread->sleep(n * SCAVENGE_WAIT_MS); } @@ -1230,7 +1230,7 @@ void Record::ShowHistory(void) void Record::queueForDelete(void) { ASSERT(state != recQueuedForDelete); - ASSERT(state == recNormal); + ASSERT(state == recNormal || state == recBuried); state = recQueuedForDelete; format->table->queueForDelete(this); } === modified file 'storage/falcon/Record.h' --- a/storage/falcon/Record.h 2009-07-08 20:05:22 +0000 +++ b/storage/falcon/Record.h 2009-07-14 14:09:58 +0000 @@ -39,7 +39,7 @@ enum RecordEncoding { longVector }; -// Record states +// Record states - must be mutually exclusive static const int recNormal = 0; // normal record //static const int recDeleted = 1; // obsolete - record has been deleted @@ -53,8 +53,9 @@ static const int recInserting = 8 //static const int recDeleting = 9; // obsolete - record is being physically deleted //static const int recPruning = 10; // obsolete - record is being pruned static const int recEndChain = 11; // end of chain for garbage collection -static const int recQueuedForDelete = 12; // Record is in purgatory. +static const int recQueuedForDelete = 12; // record is in purgatory. static const int recUpdating = 13; // record is being updated, do not chill +static const int recBuried = 14; // record has a newer version completed. Needs to be thawed. // Special value for data.record #define recordIsChilled (const char*) -1 === modified file 'storage/falcon/RecordVersion.cpp' --- a/storage/falcon/RecordVersion.cpp 2009-07-08 20:05:22 +0000 +++ b/storage/falcon/RecordVersion.cpp 2009-07-14 14:09:58 +0000 @@ -188,7 +188,15 @@ Record* RecordVersion::fetchVersion(Tran else if (recTransState->transactionId <= trans->transactionId) { if (trans->visible(recTransState, FOR_READING)) + { + char* recordData = getRecordData(); + if (recordData == recordDataIsFreed) + { + throw SQLError(INTERNAL_ERROR, "Unthawable record encountered"); + } + return (hasRecord()) ? this : NULL; + } } } @@ -424,12 +432,24 @@ char *RecordVersion::thaw() trans->release(); } - // If the record data is no longer available in the serial log, - // zap the virtual offset and restore from the data page. + // Second, try reading from the page. if (recordData == recordIsChilled) { + // The record data is no longer available in the serial log. + // Zap the virtual offset and restore from the data page. + setVirtualOffset(0); + + // If this record has been superceded/buried by another committed + // and completed record, then it is now unthawable! + + if (state == recBuried) + { + data.record = (char*) recordDataIsFreed; + return (char*) recordDataIsFreed; + } + Stream stream; Table *table = format->table; === modified file 'storage/falcon/Table.cpp' --- a/storage/falcon/Table.cpp 2009-07-08 20:05:22 +0000 +++ b/storage/falcon/Table.cpp 2009-07-14 14:09:58 +0000 @@ -3727,7 +3727,7 @@ RecordVersion* Table::allocRecordVersion // Give the scavenger thread a chance to release memory. // Increase the wait time per iteration. - Thread *thread = Thread::getThread("Database::ticker"); + Thread *thread = Thread::getThread("Table::allocRecordVersion"); thread->sleep(n * SCAVENGE_WAIT_MS); } } @@ -3761,7 +3761,7 @@ Record* Table::allocRecord(int recordNum // Give the scavenger thread a chance to release memory. // Increase the wait time per iteration. - Thread *thread = Thread::getThread("Database::ticker"); + Thread *thread = Thread::getThread("Table::allocRecord"); thread->sleep(n * SCAVENGE_WAIT_MS); } } === modified file 'storage/falcon/Transaction.cpp' --- a/storage/falcon/Transaction.cpp 2009-07-08 20:05:22 +0000 +++ b/storage/falcon/Transaction.cpp 2009-07-14 14:09:58 +0000 @@ -1609,16 +1609,67 @@ void Transaction::thawAll(void) Sync syncRec(&syncRecords,"Transaction::thawAll"); syncRec.lock(Shared); + TransId oldestActiveTransaction = database->transactionManager->findOldestInActiveList(); int totalRecords = 0; int totalThawed = 0; for (RecordVersion *record = firstRecord; record && record->nextInTrans; record = record->nextInTrans) { totalRecords++; + + // Mark prior committed records as 'buried'. This means that the data page + // contains a newer version. + + for (Record* prior = record->getPriorVersion(); + prior != NULL; + prior = prior->getPriorVersion()) + { + TransId priorTransId = prior->getTransactionId(); + if (prior->state == recBuried || oldestActiveTransaction > priorTransId) + break; + + if (transactionId > priorTransId) + prior->state = recBuried; + } + + // Thaw any record that might soon become unthawable. + if (record->isChilled()&& !record->isBase()) { totalThawed++; - record->getRecordData(); + char* recordData = record->findRecordData(); + + for (int n = 1; recordData == recordIsChilled; ++n) + { + if (transactionId < oldestActiveTransaction) + return; // Nothing needs to be thawed now. + + try + { + recordData = record->getRecordData(); + } + + catch (SQLException& exception) + { + if (exception.getSqlcode() == OUT_OF_RECORD_MEMORY_ERROR) + { + // Where error 305 (out-of-record-memory) originates, there are built in + // wait states while the scavenger is signalled. + // Give it a few more tries until it works or this committed transaction + // is older than the oldest active. + // If this gopher still cannot get record memory, just skip the thaw + // for this record. Report it as unthawable for now. + // It may become really unthawable in the future when the data page has + // a newer record and the serial log has been toggled. The client thread + // will determine that when it tried to thaw this record version. + + if (n > OUT_OF_RECORD_MEMORY_RETRIES) + recordData = (char*) recordDataIsFreed; + } + else + throw; + } + } } } --Boundary_(ID_CHGjJBmMvq0bo/b6oEnU1g) MIME-version: 1.0 Content-type: text/bzr-bundle; CHARSET=US-ASCII; name="bzr/kevin.lewis@stripped" Content-transfer-encoding: 7BIT Content-disposition: inline; filename="bzr/kevin.lewis@stripped" # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: kevin.lewis@stripped # target_branch: file:///C:/Work/bzr/Merge/mysql-6.0-falcon-team/ # testament_sha1: 5debade491252cb6deb5b25f555d7e32840f94ea # timestamp: 2009-07-14 09:10:08 -0500 # source_branch: file:///C:/Work/bzr/Merge/mysql-6.0-falcon-team/ # base_revision_id: kevin.lewis@stripped\ # pzcyq0p0f78of6o3 # # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWaegjOgAB25fgHkwef///3+v /7C////+YBB826+z16g7e0++5gumJoPec7o9L2RuO1obzPO3WVTopbJG23jdTt1CSRFPT0CpvJoU 8mTU8ozBCNPSAAyA00ZpGg0GgJMFMjKntSPNUB6gGjIyAGgBoAGIAxBNTSPSqf6I0U3pMk/VNDQz UGgDTQAAAABISISniNKZqem1Q21Nqj0jaRvSj1MQekNPKAA0GmgikIAIDSp+mmqemin6FP1TPEja iNG1PTUeUDIZDRphIkECniaAIBFP1PRE8k/U9Ke0po2o9T1AANPUD0nDgCDJgkLIEJjSYmxsQ2jY SJtC1T1WdV3p69buU36nis2sv6aXXn7MsmzOstjws3cYhNOMWESq4sF3TauMFM7EA2bDvTsUKsTN BmiwMODq+3Xm/nPqLcLdcytM4GCdn1VwrvWpLV2Umfnc4wRKN6ABmRzwr+HjOggcYWTurS4nJ6bN DpW+4jOMK6JudY7n+9fZRsbsbasIbVL7jQmpGi3XiU0cg42eQm2RWOvMjBYclezH1C7UcENsbTY2 NiObzgu2zGRtsk/LGmosLrs39YhoGaD3fPllYCKmNO8Kjhrgc6jYBZDRAYw+UDR6U2nYsK614rjq xT6PDun5QA4kRxHT0yDG7ThqTbK5ZGxtcl1Rx65xmnlzXrRNHp2MK2VD1M3y1EWa9PzUKEd9olgy cLKmVUjkmZJGKIBEM7sg6AorGBnjpPZSE5xZUOHdzQ0bS2Aoml+no+gMUq4M3PG+MRGtilMw1MN1 S1sfzY5C8stOiRfMgx0ykwvNTvhRmhNl2joLhiSoqgtA14aLJnBa1qHOEmx0PC8hyiU5VTCN6E4F pzqgerhg02/3jweUpgRFpByou4PDqfULeeI8vCemFhRYIGbBhdd4FJNcjjNUDSmVjncNthgRwFbr loeBDIxbXbMFrWuWd5o6rqsKi2yzE1QQi+SUR5Ni4QTUC7JqAwIT5M8s+UXZEvi2kTdLsXsoU7Lh 2l+WJhW+ZGdKNsa7NZGWY27uN76MJDy6lVjStEFc7ai7bsbU/c46PlbDNFWTNO52J0dkO/o4Dgz3 MefHJxcOi/uX2XajihworpdVY4bDuu3wU6+fnmvtYx8mAvMi7unevCCFlt6rIcNmZFI6+y1C6WIz aTfjp0v11o3uFOS0smjc4dOiYPZpux0TukGmsdE0jNAUu8ivKsoK+JF64imZmGVGRP4MeaNlEYYt j2vYchZN4NiQ4w9jh5OiKplcUlLjKRfIqmWG6FVePLVL8OmYgJvGKKJqOMiGkjYeJCPzni30d0KT bGMZIVUWWojKEewX0oz3mcouQiN8NoUlGuvtpC122MQirSV5AWraZFuwokkusMUmrCBQ9SXUFJK9 IuPQQSGJ1zLjAkd6Ykdb4OxdcJnGNskiJbNeVJ0GQ0hxsKkK8ieorPNI247cccbqy68jaEHuGebq F3DWLqsyVl7ZgYXlzPTxpOP0bCe8hlBOe5bqZjkGBwS42m4mAZJ+IvTflw01RPDThbmN2FEhEL0l p7MBPQhaMySuxMyalIf0alMV5SdJxeYUONhVJdaonLdshTE5A8Y41WT39a3FoRgRKtCa6xM4g4ns 1rNaFgOwGcaQtVJc6mU3qwtVqMzfac5TDDJG+BlCpyMEMvGWi4ltt9DkUNikc5sMj/rwcbTDSIt1 1KkzE16qCyOCG02MhqpVRk0N8uWuXuSmC52jKQuvqjrKxrxxdD5Xi5huOTW+aHrdd2u0tlEs9xNr N/XzWW0qTE2KwdL8j+TeZnBrXI4c5EVMUS2CUrTGlcTI6CZ1GzZejcjWuXWvmqLYtNfMiVz5paNd 2ZU2nK3YrXvdhEbiMiNQlPzVMCsqI4UF4lIRJWigjMg8kOJxQp1ZY1KpmKgRRnCAywVq+DKFhobi EcsMCwKb0blgd61FuidyN3p82rfRK0sOo4K84lgFyMFHmVt3HdjE5CXKI0JGsntqjIqcC6sERoSL dZgTI5VxLRlxeXnnRaVLDBfgRaibjNzRrebFIgSGS6ZJLotMzMYcw6IC1mmolGJoGsSqUcrWw0K6 V07Vk7lQjRzpRMEMCidpFkDeXyyxL2lgBNxD+PoQ0X7OJ8TRqrTzHJ39xw88Eedq67H6OqJNNtNv H4XnbeckNzFE9O8hoWSXe+VHyeWVjVnjgpbnFgNUkjZ5pKFfFbCDMJmbouYgzIKYid3pK0lZB5sP f2nLsjyn5JZ0TkfNciZhmIqGVyTjQ8ibBN2H76o79ZdMp6tcHB1Jn3nAmaazeW7wuXDHAamXV4be 1t758xi0TIq+GvzNKz29mXAbNiFHIJIJDsNyx6nCHSbVW/Qnp8S0gL8h/Po+RvP9E8xHSs0QJR2Y z7mgwaTe2FAMGLwPIqC8T3GB7T2jLCZ3nkQTL0ecg/T5HI+sr6w/aw1tcppr34FKwS+yq4gFeG0Q fRD739xoaE2y8boPMklhH4/fhlt/i5zpX/AaifJRLZT2rZYXZg6JX9k02vGM5WtSvUFtEIcH4Ujd CINgldqLJ0bQ1wnJIZ0d4mhgxofNcskSjbByicbKkkTPhiGE5SVcHh7sBfiOZ2HkNDYbSB19uXN5 L3hiKFnabBdvxdmSMDd+KhydS4ogWBH1HMeg6dxTatDsSNqZuMsmGumS78VlPgZ5ndSXcv3LqXm5 m2x0ZGeX5fzK8MK6BhE16encfFJDCwCkiETEIOPfuCMUB8wO3jWtVj3G3FOpxSNlJrMxuWhI5Tqr DztXtezq1mqdn0OmrLBRUDeDF7qHSHUmMIIeR0Hf7WRkiywPg7QosZlDCC5esr7iRkamtLhZqyis hJDmB83g24dJ72aoHPSERWo2nwAyi5gYNZC1XtYwaPn+L2NvaWXInn6hnEta3pSboiVJCTMBftgZ U+X1bBkBECnSRRK3Z5rdPG4FShAa+13jwYz0ik9KHFsEkNDRqNCV3CRLqH0IZSjD0nq6alh9JebT E004prXUU7k4xvCiWqQBxOIhwgLLq+VEjMZkHv5SQH2sY0JtMaaYmDE0MV+rsQXi0C2yX3y6FIuh TU9WUwoHnHWg6ty6xshMZhbPhJDD9GiJcj6zkd52lZCVF46L5fRf0hwVEaAx9PW9nALUcaKEHJD5 +Kvw6yyUWRa1Ayk59ksFQjqSZilcfNDG01UWqnX6BxsndrmNYZckVAzQ9ltGMbQcTAKuhMpB2df/ skkTQpXMfZcqb/mjYpr1dabUO/nEmD0cGFe8qCpg1FTmTLfWo1apDOajEPOVHJ5URlAoQxiV5eEq FtlK5EdCIa0SogL5DachyD0WMqUyLTbY+D/WsSI1V4Eb+ZbPRAQNwhkepbVXEQE03zGWsh+7UokX rFQWI1LUvqVKjqXszXy8FoXQehobzBwhroBQ3qzNDFSQ2ediAbSbTabbSKODHwq5fog4bhP1soMz PuoWA0dzmWG8WumAxNF7nVuBI5pVLkB8YtxQ9D3BVp0elQ1dNQugQXHt/v+L3tu1FixFZj4X52d+ iutO7ai8aA6CbTbY/K8ILIxIyckjBPrAlI8Dbbs0tFdL8PNZhq10JtVD4BoMY2DGNtppsaBgwtEt mvXBU1PMiCCGFhIkqTR8ZhpA5wFRvDDT8ozTR3FSfFJk9bxdz5XN/W4J0OsO0sg1j4NphH+vkT1Q eJCTR6QNMFktgrr4IExjURAf4amqpU8+qrNLLVOR2nzQn9PaYJCEEEQsJBCml5nEHoyD7EkcVVeG K6s8D4sbSYXrOZNhVr4LYjMN6FtQWcHxrzIrpLd6ZfS5ovkTDG6LoIYca2Kln5oIoS8idXy2FLbw +p8HsS8KzIMsRoE3ypY4I9jiRs8qy3zfejK2520XpDqFU1TD6ZB0yRWH8IUwVIUqVIHpfB6AJmMD fAF0+zJbuqKjKEsx9iVIziLB9cDNSZZZJJc4vPqb620p7BZHYwUIHx2uNBTEwvoyAnOcB/aEpBDT QXi2qGXt/G+mBHpHM5IwfWYcq8C8Al3V0rXzDlWTocne/veJaOptONFuWaprgHg7YFkMCwkIEHZc eCTbyylq8/pgOU8Iwx4E2bEiMp0qD8mjOINAhA/DrxmXtdllhwDwGxmIlvU4QVxmtizM8TKne5lw uxEHGvZGIB+tcUIls3vOGIHv7+5ZsJnWPlcEudAfoY2VhwSVd+gJsuidsL7OEg162C6vFwa9TwQO 48QjWmnp63qehcq5IxGhgxs2xDGDSahEbnIGmAvD4I7WNMabG5qYhA6uEV7BBsdsAFfIgwgQ3MMD /6R7umQlHHZ50/bVgLAfZdz6sqk6oZ9SbslswmaQpBpKgP/z9jqKo8GJBdYRZbjtHCi9IF3QHW3L /e9xmAdspESJBa0pDCxbVdFbqSbee24sF1ixehouNTghLwLYZSNmfKjAc22pDqkj3CsRNgHeBs8L LpJevHdJ0jQSoOgzMr5EtQsQQJdUljVktyYhMPaPOMZSlJJJJPZwAyfaDUaYtNkfQbdDaOQY86Nq GAt/xJEl/IcY6Nq4XcS4QYPLPEh25rMXDIa626bJrmmF567vBmxtXdNWdjFrnYfCqEgSQLkeyCIA B3nvmN9EGQKKBFVxhJJHuy2UJwMgRngmk1DecJMQViQrKETaQxCn4sE4fRHwdLTgWrRW1WVfEXBv VCIaCiAsSClrZINa6TnNp631zwIgoZwu9vYp72o6c4oepLqgwdi+fnuVv33f2RNRbBiuryRwSRPJ jA5MmojbQNerljixpi0mYXdesxB2UUZdqwXDgNpPUt3NrZmprL11QK+DgLmXuLNgueam2PCSNqEq ENVRGld2XQL7+TMNEvagyhSkwGRSLNbuck+wXutaADrPAE0YJGJ7mOEIaB2T9Dy4TCnMo2CHafI8 3l2u3qcqTcnNM3k57HkB0XQ5O2TCWJd5Tzvn8Y/wdYiFCUj/xdyRThQkKegjOgA= --Boundary_(ID_CHGjJBmMvq0bo/b6oEnU1g)--