List:Commits« Previous MessageNext Message »
From:Kevin Lewis Date:July 14 2009 8:10pm
Subject:bzr push into mysql-6.0-falcon-team branch (kevin.lewis:2754 to 2755)
Bug#45746
View as plain text  
 2755 Kevin Lewis	2009-07-14
      Bug#45746 - Gopher thread gets error 305.
      The previous patch for this bug made it highly unlikely for the
      gopher thread to get error 305-out-of-record-cache.  But it is
      still possible since thawAll() still may need to muck with the 
      record cache.
      
      So this patch makes the gopher thread retry the attempt to 
      allocate a record another 10 time.  Client threads which also 
      get this error will not retry so they will probably rollback 
      and free some memory.  In thawAll(), the gopher will also give 
      up if the transaction gets too old while this is happening.
      
      Then after 10 tries, which should take more than 22.5 seconds,
      The gopher will just give up trying to thaw that record, 
      which may never be needed anyway.  If it is needed, the client 
      thread must be able to recognize an unthawable record.
      
      If thawAll does not thaw a record, it will remain thawable
      from the serial log until the serial log toggles so that the 
      virtual offset is no longer around.  So this patch uses two 
      new indicators to identify an unthawable record.  The first is
      a new state called recBuried which indicates that the record 
      version has been replaced in the page.  This is now set in 
      thawAll(), during transaction completion.  Then later, if a 
      'buried' record cannot be thawed from the serial log, 
      RecordVersion::thaw() does not attempt to thaw it from the 
      page.  It just marks it as unthawable by setting 
      Record::data.record = recordDataIsFreed.
      
      Then the client thread in fetchVersion() must throw an error 
      indicating that a visible record cannot be thawed.
     @ storage/falcon/Record.cpp
        Bug#45746 - account for new state recBuried
     @ storage/falcon/Record.h
        Bug#45746 - new state recBuried
     @ storage/falcon/RecordVersion.cpp
        Bug#45746 - fetchVersion() should recognize an unthawable record
        It throws an internal error with an appropriate message.
        RecordVersion::thaw() now avoids thawing a record from the data 
        page that has been superceded.
     @ storage/falcon/Table.cpp
        code cleanup
     @ storage/falcon/Transaction.cpp
        Bug#45746 - thawAll() is enhanced to attempt to thaw arecord 10
        time if the error is 305-out-of-record-memory.  It will also
        quit early if the transaction becomes too old.  Then if 10 loops
        occur, it will just ignore that record, hoping that it is not 
        actually needed before the serial log toggles and it becomes 
        unthawable.
        
        thawAll also now marks prior records that have been superceded
        by these completed records as recBuried.
        the data page, or 'buried'.

    modified:
      storage/falcon/Record.cpp
      storage/falcon/Record.h
      storage/falcon/RecordVersion.cpp
      storage/falcon/Table.cpp
      storage/falcon/Transaction.cpp
 2754 Kevin Lewis	2009-07-12 [merge]
      merge

    modified:
      storage/falcon/Cache.cpp
      storage/falcon/DataPage.cpp
      storage/falcon/Dbb.cpp
      storage/falcon/Dbb.h
      storage/falcon/IO.cpp
=== modified file 'storage/falcon/Record.cpp'
--- a/storage/falcon/Record.cpp	2009-07-08 20:05:22 +0000
+++ b/storage/falcon/Record.cpp	2009-07-14 14:09:58 +0000
@@ -1094,7 +1094,7 @@ char* Record::allocRecordData(int length
 			// Give the scavenger thread a chance to release memory.
 			// Increase the wait time per iteration.
 
-			Thread *thread = Thread::getThread("Database::ticker");
+			Thread *thread = Thread::getThread("Record::allocRecordData");
 			thread->sleep(n * SCAVENGE_WAIT_MS);
 			}
 	
@@ -1230,7 +1230,7 @@ void Record::ShowHistory(void)
 void Record::queueForDelete(void)
 {
 	ASSERT(state != recQueuedForDelete);
-	ASSERT(state == recNormal);
+	ASSERT(state == recNormal || state == recBuried);
 	state = recQueuedForDelete;
 	format->table->queueForDelete(this);
 }

=== modified file 'storage/falcon/Record.h'
--- a/storage/falcon/Record.h	2009-07-08 20:05:22 +0000
+++ b/storage/falcon/Record.h	2009-07-14 14:09:58 +0000
@@ -39,7 +39,7 @@ enum RecordEncoding {
 	longVector
 	};
 
-// Record states
+// Record states - must be mutually exclusive
 
 static const int recNormal          =  0;	// normal record
 //static const int recDeleted         =  1;	// obsolete - record has been deleted
@@ -53,8 +53,9 @@ static const int recInserting       =  8
 //static const int recDeleting        =  9;	// obsolete - record is being physically deleted
 //static const int recPruning         = 10;	// obsolete - record is being pruned
 static const int recEndChain          = 11;	// end of chain for garbage collection
-static const int recQueuedForDelete = 12;	// Record is in purgatory.
+static const int recQueuedForDelete = 12;	// record is in purgatory.
 static const int recUpdating        = 13;	// record is being updated, do not chill
+static const int recBuried          = 14;	// record has a newer version completed.  Needs to be thawed.
 
 // Special value for data.record
 #define recordIsChilled     (const char*) -1

=== modified file 'storage/falcon/RecordVersion.cpp'
--- a/storage/falcon/RecordVersion.cpp	2009-07-08 20:05:22 +0000
+++ b/storage/falcon/RecordVersion.cpp	2009-07-14 14:09:58 +0000
@@ -188,7 +188,15 @@ Record* RecordVersion::fetchVersion(Tran
 		else if (recTransState->transactionId <= trans->transactionId)
 			{
 			if (trans->visible(recTransState, FOR_READING))
+				{
+				char* recordData = getRecordData();
+				if (recordData == recordDataIsFreed)
+					{
+					throw SQLError(INTERNAL_ERROR, "Unthawable record encountered");
+					}
+
 				return (hasRecord()) ? this : NULL;
+				}
 			}
 		}
 
@@ -424,12 +432,24 @@ char *RecordVersion::thaw()
 		trans->release();
 		}
 
-	// If the record data is no longer available in the serial log, 
-	// zap the virtual offset and restore from the data page.
+	// Second, try reading from the page.
 
 	if (recordData == recordIsChilled)
 		{
+		// The record data is no longer available in the serial log.
+		// Zap the virtual offset and restore from the data page.
+
 		setVirtualOffset(0);
+
+		// If this record has been superceded/buried by another committed
+		// and completed record, then it is now unthawable!
+
+		if (state == recBuried)
+			{
+			data.record = (char*) recordDataIsFreed;
+			return (char*) recordDataIsFreed;
+			}
+
 		Stream stream;
 		Table *table = format->table;
 

=== modified file 'storage/falcon/Table.cpp'
--- a/storage/falcon/Table.cpp	2009-07-08 20:05:22 +0000
+++ b/storage/falcon/Table.cpp	2009-07-14 14:09:58 +0000
@@ -3727,7 +3727,7 @@ RecordVersion* Table::allocRecordVersion
 			// Give the scavenger thread a chance to release memory.
 			// Increase the wait time per iteration.
 
-			Thread *thread = Thread::getThread("Database::ticker");
+			Thread *thread = Thread::getThread("Table::allocRecordVersion");
 			thread->sleep(n * SCAVENGE_WAIT_MS);
 			}
 		}
@@ -3761,7 +3761,7 @@ Record* Table::allocRecord(int recordNum
 			// Give the scavenger thread a chance to release memory.
 			// Increase the wait time per iteration.
 
-			Thread *thread = Thread::getThread("Database::ticker");
+			Thread *thread = Thread::getThread("Table::allocRecord");
 			thread->sleep(n * SCAVENGE_WAIT_MS);
 			}
 		}

=== modified file 'storage/falcon/Transaction.cpp'
--- a/storage/falcon/Transaction.cpp	2009-07-08 20:05:22 +0000
+++ b/storage/falcon/Transaction.cpp	2009-07-14 14:09:58 +0000
@@ -1609,16 +1609,67 @@ void Transaction::thawAll(void)
 	Sync syncRec(&syncRecords,"Transaction::thawAll");
 	syncRec.lock(Shared);
 
+	TransId oldestActiveTransaction = database->transactionManager->findOldestInActiveList();
 	int totalRecords = 0;
 	int totalThawed = 0;
 
 	for (RecordVersion *record = firstRecord; record && record->nextInTrans; record = record->nextInTrans)
 		{
 		totalRecords++;
+
+		// Mark prior committed records as 'buried'. This means that the data page
+		// contains a newer version.  
+
+		for (Record* prior = record->getPriorVersion();
+			 prior != NULL;
+			 prior = prior->getPriorVersion())
+			{
+			TransId priorTransId = prior->getTransactionId();
+			if (prior->state == recBuried || oldestActiveTransaction > priorTransId)
+				break;
+
+			if (transactionId > priorTransId)
+				prior->state = recBuried;
+			}
+
+		// Thaw any record that might soon become unthawable.
+
 		if (record->isChilled()&& !record->isBase())
 			{
 			totalThawed++;
-			record->getRecordData();
+			char* recordData = record->findRecordData();
+
+			for (int n = 1; recordData == recordIsChilled; ++n)
+				{
+				if (transactionId < oldestActiveTransaction)
+					return;  // Nothing needs to be thawed now.
+
+				try
+					{
+					recordData = record->getRecordData();
+					}
+
+				catch (SQLException& exception)
+					{
+					if (exception.getSqlcode() == OUT_OF_RECORD_MEMORY_ERROR)
+						{
+						// Where error 305 (out-of-record-memory) originates, there are built in 
+						// wait states while the scavenger is signalled.  
+						// Give it a few more tries until it works or this committed transaction 
+						// is older than the oldest active.
+						// If this gopher still cannot get record memory, just skip the thaw 
+						// for this record.  Report it as unthawable for now.
+						// It may become really unthawable in the future when the data page has
+						// a newer record and the serial log has been toggled.  The client thread
+						// will determine that when it tried to thaw this record version.
+
+						if (n > OUT_OF_RECORD_MEMORY_RETRIES)
+							recordData = (char*) recordDataIsFreed;
+						}
+					else
+						throw;
+					}
+				}
 			}
 		}
 


Attachment: [text/bzr-bundle] bzr/kevin.lewis@sun.com-20090714140958-pxo649nosfeib1x9.bundle
Thread
bzr push into mysql-6.0-falcon-team branch (kevin.lewis:2754 to 2755)Bug#45746Kevin Lewis15 Jul