List:Commits« Previous MessageNext Message »
From:Kevin Lewis Date:June 10 2009 7:38am
Subject:bzr push into mysql-6.0-falcon-team branch (kevin.lewis:2717 to 2722)
Bug#36631
View as plain text  
 2722 Kevin Lewis	2009-06-10
      Bug#36631 - Make sure that SerialLogControl::currentLength is always set correctly.  If it is too short and a buffer needs to be re-read, not enough of the serial log will be read back inot the buffer. When it gets to the end of the current length and calls SerialLogControl::nextRecord() it will hit an assert when it reads the previous contents of that buffer, which is now junk.  The problem is that the block length indicates a larger window than the currentLength.  
      
      SerialLogControl::currentLength is now updated in each location that the current writePtr is extended.  This is three more places than before.
      
      In addition, some comments and some just-in-case code is added to SRLUpdateRecords::thaw().  Since this thaw is not calling setWindow(), but only doing a subset of that function, the task of clearing anddeactivating the old inputWindow that is done in setWindow() is repeated here.

    modified:
      storage/falcon/SRLUpdateRecords.cpp
      storage/falcon/SerialLog.cpp
 2721 John H. Embretsen	2009-06-04
      Fix accidental change to falcon_team test suite's "experimental" status in PB2 committed with revision 
      alik@stripped. See bug 43982 for further justification.

    modified:
      mysql-test/collections/mysql-6.0-falcon-.push
 2720 Kevin Lewis	2009-06-04 [merge]
      Merge

    modified:
      libservices/thd_alloc_service.c
      mysql-test/lib/My/SafeProcess/safe_process_win.cc
       2718.1.1 John H. Embretsen	2009-06-04 [merge]
                Merge mysql-6.0-falcon --> mysql-6.0-falcon-team.
                Includes fix for MTR/Pushbuild/Windows bug (44775).

        modified:
          libservices/thd_alloc_service.c
          mysql-test/lib/My/SafeProcess/safe_process_win.cc
       2707.1.4 John H. Embretsen	2009-06-03 [merge]
                Cherrypick (merge) fix for bug 44775 (MTR fails to bootstrap mysqld on Windows in Pushbuild 2)
                from mysql-6.0-mtr to mysql-6.0-falcon.

        modified:
          mysql-test/lib/My/SafeProcess/safe_process_win.cc
          2651.27.1 Vladislav Vaintroub	2009-05-15 [merge]
                    merge fix for 44775

            modified:
              mysql-test/lib/My/SafeProcess/safe_process_win.cc
         2497.899.2 Vladislav Vaintroub	2009-05-15 [merge]
                    merge

            modified:
              mysql-test/lib/mtr_cases.pm
              mysql-test/mysql-test-run.pl
         2497.899.1 Vladislav Vaintroub	2009-05-14
                    Bug #44775 MTR fails to bootstrap mysqld on Windows in Pushbuild 2.
                    
                    Suspected reason for the failure is that safe_process.exe already runs in a job that does not allow breakaways. 
                    The fix is to use a fallback -  make newly created process the root of the new process group. This allows to kill process together with descendants via GenerateConsoleCtrlEvent (CTRL_BREAK_EVENT, pid)

            modified:
              mysql-test/lib/My/SafeProcess/safe_process_win.cc
       2707.1.3 Vladislav Vaintroub	2009-05-29
                Fix typo (breaks windows build)

        modified:
          libservices/thd_alloc_service.c
       2707.1.2 John H. Embretsen	2009-05-20
                Fixed .bzr-mysql/default.conf after merging falcon-team --> falcon.

        modified:
          .bzr-mysql/default.conf
 2719 Kevin Lewis	2009-06-04
      Bug#43344 - One more cleanup.  RecordVersion::hasData() should act like RecordVersion::getRecordData() by testing the recordData returned from thaw() instead of calling isChilled() which tests data.record again.  The record can immediately get re-chilled.  And that actually happened in a pushbuld test.

    modified:
      storage/falcon/RecordVersion.cpp
 2718 Kevin Lewis	2009-06-03
      Bug#44911 - More follow-on changes.  
      This assures that there will not be a infinite loop in the case where a lock record is returned from Table::fetch and the transaction becomes CommittedVisible soon after.  This should only occur once.  The next time, it has to be a different lock record.  If the same record is read again, something is fataly wrong.  It happened once while I was making some changes for 43344 because of a mistake. So instead of looping infinitely reading the same record, this patch will issue a fatal error.

    modified:
      storage/falcon/Table.cpp
 2717 Kevin Lewis	2009-06-03
      Bug#43961 & Bug#44223  Both of these bugs are caused by a record that gets queued for delete while still chained up to the transaction.  Then during Transaction::rolback  (44223)   a record to rollback is queued for delete a second time.
      Or during Transaction::rollbackSavepoint  (43961), the code assumes that this record on the transaction and savepoint is still the base record since it has Record::superceded=false, and tries to call insertIntoTree to rerplace it with its prior.  But this record is not the base record either.
      
      This can happen in Table::update(Transaction * transaction, Record *orgRecord, Stream *stream);
      If this update replaces a lock record with an updated record and an exception occurred, it may or may not take the record off the transaction and the tree, but it would always queue the lock record for delete.  This code has gone through some iterations lately.  But in the most recent code, I was able to reporduce 44223. 
      
      This patch cleans it up even more by distinguishing whether a record is actually inserted into the tree and added to the transaction.  It undoes those actions only if they were done. Then if the update started with a lock record, that recordversion will remain on the tree with superceded=false and it will not be queued for delete.  It correctly uses the 'wasLock' variable instead of the 'updated' or 'wasUpdated' flag.

    modified:
      storage/falcon/Table.cpp
=== modified file 'libservices/thd_alloc_service.c'
--- a/libservices/thd_alloc_service.c	2009-05-06 18:17:49 +0000
+++ b/libservices/thd_alloc_service.c	2009-05-29 19:05:24 +0000
@@ -14,4 +14,4 @@
    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA */
 
 #include <service_versions.h>
-SERVICE_VERSION thd_alloc_servicee= (void*)VERSION_thd_alloc;
+SERVICE_VERSION thd_alloc_service= (void*)VERSION_thd_alloc;

=== modified file 'mysql-test/collections/mysql-6.0-falcon-.push'
--- a/mysql-test/collections/mysql-6.0-falcon-.push	2009-04-28 08:13:50 +0000
+++ b/mysql-test/collections/mysql-6.0-falcon-.push	2009-06-04 14:00:27 +0000
@@ -4,5 +4,5 @@ perl mysql-test-run.pl --comment=embedde
 perl mysql-test-run.pl --comment=rpl_binlog_row                   --mysqld=--binlog-format=row                                                  --suite=rpl,binlog                                                        --experimental=collections/default.experimental --timer --force 
 perl mysql-test-run.pl --comment=funcs_1                                                                                                        --suite=funcs_1                                                           --experimental=collections/default.experimental --timer --force 
 perl mysql-test-run.pl --comment=ps_stm_threadpool --ps-protocol  --mysqld=--binlog-format=statement --mysqld=--thread-handling=pool-of-threads --suite=main,backup,backup_engines,backup_ptr,binlog,federated,rpl,maria  --experimental=collections/default.experimental --timer --force 
-perl mysql-test-run.pl --comment=falcon                                                                                                         --suite=falcon                                                            --experimental=collections/default.experimental --timer --force 
-perl mysql-test-run.pl --comment=falcon_exp                                                                                                     --suite=falcon_team                                                                                                       --timer --force
+perl mysql-test-run.pl --comment=falcon --suite=falcon --experimental=collections/default.experimental --timer --force 
+perl mysql-test-run.pl --comment=falcon_exp --suite=falcon_team --experimental=collections/falcon_team.experimental --timer --force

=== modified file 'mysql-test/lib/My/SafeProcess/safe_process_win.cc'
--- a/mysql-test/lib/My/SafeProcess/safe_process_win.cc	2009-02-09 18:24:48 +0000
+++ b/mysql-test/lib/My/SafeProcess/safe_process_win.cc	2009-05-14 19:56:53 +0000
@@ -259,22 +259,37 @@ int main(int argc, const char** argv )
     the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag, making sure it will be
     terminated when the last handle to it is closed(which is owned by
     this process).
+
+    If breakaway from job fails on some reason, fallback is to create a
+    new process group. Process groups also allow to kill process and its 
+    descedants, subject to some restrictions (processes have to run within
+    the same console,and must not ignore CTRL_BREAK)
   */
-  if (CreateProcess(NULL, (LPSTR)child_args,
+  DWORD create_flags[]= {CREATE_BREAKAWAY_FROM_JOB, CREATE_NEW_PROCESS_GROUP, 0};
+  BOOL process_created= FALSE;
+  BOOL jobobject_assigned= FALSE;
+
+  for (int i=0; i < sizeof(create_flags)/sizeof(create_flags[0]); i++)
+  { 
+    process_created= CreateProcess(NULL, (LPSTR)child_args,
                     NULL,
                     NULL,
                     TRUE, /* inherit handles */
-                    CREATE_SUSPENDED | CREATE_BREAKAWAY_FROM_JOB,
+                    CREATE_SUSPENDED | create_flags[i],
                     NULL,
                     NULL,
                     &si,
-                    &process_info) == 0)
-    die("CreateProcess failed");
+                    &process_info);
+    if (process_created)
+    {
+     jobobject_assigned= AssignProcessToJobObject(job_handle, process_info.hProcess);
+     break;
+    }
+  }
 
-  if (AssignProcessToJobObject(job_handle, process_info.hProcess) == 0)
+  if (!process_created)
   {
-    TerminateProcess(process_info.hProcess, 200);
-    die("AssignProcessToJobObject failed");
+    die("CreateProcess failed");
   }
   ResumeThread(process_info.hThread);
   CloseHandle(process_info.hThread);
@@ -312,6 +327,13 @@ int main(int argc, const char** argv )
     message("TerminateJobObject failed");
   CloseHandle(job_handle);
   message("Job terminated and closed");
+
+  if (!jobobject_assigned)
+  {
+    GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, process_info.dwProcessId);
+    TerminateProcess(process_info.hProcess, 202);
+  }
+
   if (wait_res != WAIT_OBJECT_0 + CHILD)
   {
     /* The child has not yet returned, wait for it */

=== modified file 'storage/falcon/RecordVersion.cpp'
--- a/storage/falcon/RecordVersion.cpp	2009-06-03 18:15:32 +0000
+++ b/storage/falcon/RecordVersion.cpp	2009-06-04 12:55:34 +0000
@@ -548,7 +548,7 @@ bool RecordVersion::hasRecord(void)
 	if (isChilled())
 		{
 		recordData = thaw();
-		ASSERT(!isChilled());
+		ASSERT(recordData != recordIsChilled);
 		return (recordData != NULL);
 		}
 

=== modified file 'storage/falcon/SRLUpdateRecords.cpp'
--- a/storage/falcon/SRLUpdateRecords.cpp	2009-06-03 18:15:32 +0000
+++ b/storage/falcon/SRLUpdateRecords.cpp	2009-06-10 07:36:02 +0000
@@ -73,7 +73,7 @@ char* SRLUpdateRecords::thaw(RecordVersi
 	// activate the window, reading from disk if necessary
 
 	SerialLogWindow *window = log->findWindowGivenOffset(record->getVirtualOffset());
-	
+
 	// If the serial log window is no longer available, then this virtualOffset is 
 	// no longer any good.  Reset it and return.
 
@@ -85,9 +85,20 @@ char* SRLUpdateRecords::thaw(RecordVersi
 	Sync sync(&log->syncWrite, "SRLUpdateRecords::thaw");
 	sync.lock(Exclusive);
 
-	// Return pointer to record data
+	// Instead of calling setWindow(), we only set input and inputEnd.
+	// Do not even assign our local 'window' to inputWindow because we will
+	// deactivate it here.
+
+	if (control->inputWindow && control->inputWindow != window)
+		{
+		control->inputWindow->deactivateWindow();
+		control->inputWindow = NULL;
+		}
+
+	// Set the input and inputEnd pointers to the record we are thawing.
 
-	control->input = window->buffer + (record->getVirtualOffset() - window->virtualOffset);
+	uint64 recordOffset = record->getVirtualOffset() - window->virtualOffset;
+	control->input = window->buffer + recordOffset;
 	control->inputEnd = window->bufferEnd;
 
 	// Get section id, record id and data length written. Input pointer will be at

=== modified file 'storage/falcon/SerialLog.cpp'
--- a/storage/falcon/SerialLog.cpp	2009-05-12 14:35:40 +0000
+++ b/storage/falcon/SerialLog.cpp	2009-06-10 07:36:02 +0000
@@ -373,7 +373,7 @@ void SerialLog::recover()
 	// probing the log files, open a window on the first block
 	// we're going to use for recovery.  That should work, but
 	// if it doesn't, complain.
-    
+
 	lastWindow->deactivateWindow();
 	SerialLogWindow *window = findWindowGivenBlock(readBlockNumber);
 	
@@ -544,6 +544,8 @@ void SerialLog::overflowFlush(void)
 	// OK, we're going to do some writing.  Start by locking the serial log
 
 	*writePtr++ = srlEnd | LOW_BYTE_FLAG;
+	if (writePtr > writeWindow->buffer + writeWindow->currentLength)
+		writeWindow->currentLength = (int) (writePtr - writeWindow->buffer);
 	writeBlock->length = (int) (writePtr - (UCHAR*) writeBlock);
 	writeWindow->setLastBlock(writeBlock);
 	lastReadBlock = writeBlock->readBlockNumber = getReadBlock();
@@ -638,6 +640,8 @@ uint64 SerialLog::flush(bool forceNewWin
 	ASSERT(writer == NULL || writer == thread);
 	writer = thread;
 	*writePtr++ = srlEnd | LOW_BYTE_FLAG;
+	if (writePtr > writeWindow->buffer + writeWindow->currentLength)
+		writeWindow->currentLength = (int) (writePtr - writeWindow->buffer);
 	writeBlock->length = (int) (writePtr - (UCHAR*) writeBlock);
 	lastReadBlock = writeBlock->readBlockNumber = getReadBlock();
 
@@ -795,6 +799,8 @@ void SerialLog::putData(uint32 length, c
 	UCHAR recordCode = *recordStart;
 	writeBlock->length = (int) (recordStart - (UCHAR*) writeBlock);
 	writePtr = recordStart;
+	if (writePtr > writeWindow->buffer + writeWindow->currentLength)
+		writeWindow->currentLength = (int) (writePtr - writeWindow->buffer);
 	overflowFlush();
 
 	while (writePtr + length + tailLength >= writeWarningTrack)

=== modified file 'storage/falcon/Table.cpp'
--- a/storage/falcon/Table.cpp	2009-06-03 20:49:27 +0000
+++ b/storage/falcon/Table.cpp	2009-06-03 21:01:11 +0000
@@ -3498,6 +3498,7 @@ Record* Table::fetchForUpdate(Transactio
 	//  Find the record that will be locked
 
 	int recordNumber = source->recordNumber;
+	Record* prevSource = NULL;
 
 	// If we already have this locked or updated, get the active version
 
@@ -3555,7 +3556,18 @@ Record* Table::fetchForUpdate(Transactio
 			case CommittedVisible:
 				{
 				if (source->isALock())
+					{
+					// A lock record may heve been read above, and before we called getRelativeState,
+					// the transactino may have committed a change.  It could be visible for 
+					// read committed and write committed isolations.  But if the same lock record 
+					// is read again, it could cause a hang and it should not ever happen.  
+					// So prevent an unlikely hang with a FATAL message.
+
+					if (prevSource == source)
+						FATAL("Table::fetchForUpdate found a committed visible lock record.");
+
 					break;  // need to re-fetch the base record
+					}
 
 				if (source->isDeleted())
 					{
@@ -3605,6 +3617,7 @@ Record* Table::fetchForUpdate(Transactio
 				throw SQLError(RUNTIME_ERROR, "unexpected transaction state %d", state);
 			}
 
+		prevSource = source;  // Don't ever folow this pointer!
 		source->release(REC_HISTORY);
 
 		source = fetch(recordNumber);


Attachment: [text/bzr-bundle] bzr/kevin.lewis@sun.com-20090610073602-wlfh2ajbgxeykcbg.bundle
Thread
bzr push into mysql-6.0-falcon-team branch (kevin.lewis:2717 to 2722)Bug#36631Kevin Lewis10 Jun