bzr commit into mysql-6.0-falcon-team branch (vvaintroub:2875) Bug#38186, Bug#39789

[Vladislav Vaintroub]

#At file:///media/disk/vmware-tools-distrib/mysql-6.0-falcon-team/

 2875 Vladislav Vaintroub	2008-10-21
      Bug#39789: Falcon recovery failure after several CREATE + 
      DROP TABLESPACE
      
      Solved in this patch:
      
      -Drop tablespace in REDO phase removed datafiles without 
      taking into account that some of them should stay after the 
      recovery. In this case, even if metadata for tablespace 
      would be present after recovery, the data is lost together 
      with the file.
      
      Typical scenario for such failed recovery would be something
      like
      
      create tablespace A add datafile 'f' engine falcon;
      drop tablespace A;
      create tablespace B add datafile 'f' engine falcon;
      create table T tablespace B;
      
      Recovery for "drop tablespace A" would delete file used
      by tablespace B and invalidate table T.
      
      Solution for this problem is to calculate whether datafiles 
      should stay after recovery or can be deleted. Calculation done
      in the pass1 of recovery, deletion in the REDO phase.
      
      While fixing this, other smaller problems were fixed
      - failure to recover, when the database crashed after datafile 
      had been created but before header was written.
      - redoCreateTableSpace did not attempt to create tablespaces, 
      but only opened existing ones.
      - fixed a long-standing problem that tablespace file was present
      even after "drop tablespace" successfully completed. This was
      previously worked around in Bug#38186
modified:
  storage/falcon/IO.cpp
  storage/falcon/IOx.h
  storage/falcon/SRLCreateTableSpace.cpp
  storage/falcon/SRLDropTableSpace.cpp
  storage/falcon/SerialLog.cpp
  storage/falcon/SerialLog.h
  storage/falcon/SerialLogFile.cpp
  storage/falcon/StorageHandler.cpp
  storage/falcon/TableSpace.cpp
  storage/falcon/TableSpace.h
  storage/falcon/TableSpaceManager.cpp
  storage/falcon/TableSpaceManager.h

per-file messages:
  storage/falcon/IO.cpp
    Support for removing open files on Windows, in a similar how
    Unix unlink() works.
    
    New function winOpen()- a posix-style wrapper for CreateFile that calls underlying CreateFile with FILE_SHARE_DELETE
    
    New function winUnlink().
    when deleting file, first move it to unique name, open
    with FILE_FLAG_DELETE_ON_CLOSE and close the handle. File will
    disappear after the last handle to it is closed. This is the only safe method to remove an open file.
    
    - New function IO::minTableSpaceSize() used in recovery (files smaller then this can sefely be deleted and recreated)
    
    - new function dupFile() - wrapper for dup()
    
    - new function isSameFile() - determine whether 2 paths refer to the same file
  storage/falcon/SRLCreateTableSpace.cpp
    during create/drop tablespace recovery in pass1, calculate the final state of each datafile , whether it should stay or can be deleted
  storage/falcon/SRLDropTableSpace.cpp
    During create/drop tablespace recovery in pass1, calculate the final state of each datafile , whether it should stay or can be deleted.
    Delete file in redo phase if it can safely be deleted.
  storage/falcon/SerialLog.cpp
    Track creation and removal of datafiles during recovery of createand drop tablespace
  storage/falcon/SerialLog.h
    Track creation and removal of datafiles during recovery of createand drop tablespace
  storage/falcon/SerialLogFile.cpp
    initial file size of serial log file is mkeant to be 8K at least
  storage/falcon/StorageHandler.cpp
    Remove waitForPendingDrop hack
  storage/falcon/TableSpace.cpp
    Depending on parameter passed, TableSpace::open either open()'s
    or dup()'s the datafile.
  storage/falcon/TableSpace.h
    An extra parameter for open - IO can be passed in this case file descriptor is duplicated. It is necessary because during the recovery phase we can have multiple tablespaces all poining to the same file. In this case we use dup()  rather then open() - on Windows multiple open's()  for the same file will fail with ERROR_SHARING_VIOLATION, as the datafiles are opened without FILE_SHARE_READ|FILE_SHARE_WRITE.
  storage/falcon/TableSpaceManager.cpp
    Create/Drop tablespace recovery
    - Tablespace::bootstrap might create a missing datafile.
    It is necessary because datafile can be physically deleted before metadata stored in system table disappear.
    - Tablespace::redoCreateTableSpace() will create datafile that is missing
    - waitForPendingDrop() hack is removed
    - extra parameter for expungeTableSpace(). expungeTableSpace it is used in redo phase of recovery and the file should stay after the recovery, file will not be removed, it is just closed.
  storage/falcon/TableSpaceManager.h
    - extra parameter for expungeTableSpace()
    - remove waitForPendingDrop hack
=== modified file 'storage/falcon/IO.cpp'
--- a/storage/falcon/IO.cpp	2008-09-11 10:56:00 +0000
+++ b/storage/falcon/IO.cpp	2008-10-21 20:05:54 +0000
@@ -117,6 +117,19 @@ static char baseDir[PATH_MAX+1]={0};
 bool deleteFilesOnExit = false;
 bool inCreateDatabase = false;
 
+#ifdef _WIN32
+static int winUnlink(const char *file);
+static int winOpen(const char *filename, int flags,...);
+#endif
+
+#ifdef _WIN32
+#define POSIX_OPEN_FILE winOpen
+#define POSIX_UNLINK_FILE winUnlink
+#else
+#define POSIX_OPEN_FILE   ::open
+#define POSIX_UNLINK_FILE ::unlink
+#endif
+
 #ifdef _DEBUG
 #undef THIS_FILE
 static const char THIS_FILE[]=__FILE__;
@@ -186,7 +199,7 @@ bool IO::openFile(const char * name, boo
 	ASSERT(!inCreateDatabase);
 
 	fileName = getPath(name);
-	fileId = ::open (fileName, (readOnly) ? (O_RDONLY | O_BINARY) : (O_RDWR | O_BINARY));
+	fileId = POSIX_OPEN_FILE(fileName, (readOnly) ? (O_RDONLY | O_BINARY) : (O_RDWR | O_BINARY));
 
 	if (fileId < 0)
 		{
@@ -219,6 +232,70 @@ bool IO::openFile(const char * name, boo
 	return fileId != -1;
 }
 
+/* 
+ Check if two filenames refer to the same existing file.
+ Simple string comparison would not yield correct results, because of symbolic 
+ links  and filename canonicalization issues.
+*/
+bool IO::isSameFile (const char *filename1, const char *filename2)
+{
+#ifndef _WIN32
+	struct stat stat1;
+	struct stat stat2;
+	if (stat(filename1, &stat1) == 0 && stat(filename2, &stat2) == 0)
+		return (stat1.st_dev == stat2.st_dev && 
+			stat1.st_ino == stat2.st_ino);
+	return false;
+#else
+	HANDLE hFile1 = CreateFile(filename1, FILE_READ_ATTRIBUTES,
+		FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
+		NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,NULL);
+
+	if(hFile1 == INVALID_HANDLE_VALUE)
+		return false;
+
+	HANDLE hFile2 = CreateFile(filename1, FILE_READ_ATTRIBUTES,
+		FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
+		NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,NULL);
+
+	if(hFile2 == INVALID_HANDLE_VALUE)
+		{
+		CloseHandle(hFile1);
+		return false;
+		}
+	BY_HANDLE_FILE_INFORMATION info1, info2;
+	bool retval;
+
+	if (GetFileInformationByHandle(hFile1, &info1) &&
+		GetFileInformationByHandle(hFile2, &info2))
+		{
+		retval = (info1.dwVolumeSerialNumber == info2.dwVolumeSerialNumber
+			&& info1.nFileIndexHigh == info2.nFileIndexHigh
+			&& info1.nFileSizeLow == info2.nFileSizeLow );
+		}
+	else
+		retval = false;
+
+	CloseHandle(hFile1);
+	CloseHandle(hFile2);
+
+	return retval;
+#endif
+}
+
+void IO::dupFile(IO *otherFile)
+{
+	fileName = otherFile->fileName;
+	fileId = dup(otherFile->fileId);
+	isReadOnly = false;
+	created = false;
+	if (fileId < 0)
+		{
+		throw SQLEXCEPTION (IO_ERROR, "can't dup file \"%s\", fileId %d: %s (%d)", 
+							fileName.getString(), fileId, strerror (errno), errno);
+		}
+}
+
 void IO::setWriteFlags(int fileId, bool *forceFsync)
 {
 #ifndef _WIN32
@@ -241,7 +318,7 @@ bool IO::createFile(const char *name)
 	Log::debug("IO::createFile: creating file \"%s\"\n", name);
 
 	fileName = getPath(name);
-	fileId = ::open (fileName.getString(),O_CREAT | O_RDWR | O_RANDOM | O_EXCL | O_BINARY,
+	fileId = POSIX_OPEN_FILE (fileName.getString(),O_CREAT | O_RDWR | O_RANDOM | O_EXCL | O_BINARY,
 						S_IREAD | S_IWRITE | S_IRGRP | S_IWGRP);
 
 
@@ -377,7 +454,12 @@ void IO::readHeader(Hdr * header)
 	n = ::read (fileId, buffer, maxPageSize);
 
 	if (n < (int) sizeof (Hdr))
-		FATAL ("read error on database header");
+		{
+		throw SQLError(IO_ERROR,
+					"read header returned %d, filename %s, errno %d",
+					n, (const char*) fileName, errno);
+		}
+
 
 	Hdr* hdr = (Hdr*) buffer;
 	if (falcon_checksums && hdr->pageSize <= n)
@@ -532,6 +614,11 @@ int IO::fileStat(const char *fileName, s
 	return(retCode);
 }
 
+int IO::minTableSpaceSize()
+{
+	return sizeof(Hdr);
+}
+
 void IO::write(uint32 length, const UCHAR *data)
 {
 	uint32 len = ::write (fileId, data, length);
@@ -557,6 +644,7 @@ void IO::writeHeader(Hdr *header)
 void IO::deleteFile()
 {
 	deleteFile(fileName);
+	fileName="";
 }
 
 int IO::pread(int64 offset, int length, UCHAR* buffer)
@@ -675,10 +763,132 @@ void IO::sync(void)
 		traceOperation(TRACE_SYNC_END);
 }
 
+#ifdef _WIN32
+#define FALCON_DELETED_FILE "fdf"
+
+/*
+	The only safe way to delete file on Windows without invalidating open file 
+	handles is that:
+	- rename file to be deleted to a unique temporary name.
+	- open file it with FILE_FLAG_DELETE_ON_CLOSE
+	- close the file handle
+	Temp file will disappear as soon as last handle on it is closed.
+	This works only if files are opened with FILE_SHARE_DELETE flag.
+*/
+
+static int winUnlink(const char *file)
+{
+	DWORD attributes = GetFileAttributes(file);
+
+	// Bail out, if file does not exist.
+	if (attributes == INVALID_FILE_ATTRIBUTES)
+		return -1;
+
+	// If file is a symbolic link, just delete the link, but not the link target
+	if (attributes & FILE_ATTRIBUTE_REPARSE_POINT)
+		{
+		if(DeleteFile(file))
+			return 0;
+		return -1;
+		}
+
+	// Rename the file to unique name, then open with FILE_FLAG_DELETE_ON_CLOSE
+	// and close.
+	char  tmpDir[MAX_PATH];
+	strncpy(tmpDir, file, sizeof(tmpDir)-1);
+	char *p = strrchr(tmpDir ,SEPARATOR);
+	if (p)
+		*p = 0;
+	else
+		strcpy(tmpDir,".");
+
+	char  tmpFile[MAX_PATH];
+	if (GetTempFileName(tmpDir, FALCON_DELETED_FILE, 0, tmpFile))
+		{
+		if (MoveFileEx(file, tmpFile, MOVEFILE_REPLACE_EXISTING))
+			{
+			HANDLE hFile = CreateFile(tmpFile, 0,
+				FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
+				NULL, OPEN_EXISTING, FILE_FLAG_DELETE_ON_CLOSE, NULL);
+
+			if (hFile != INVALID_HANDLE_VALUE)
+				{
+				CloseHandle(hFile);
+				return 0;
+				}
+			}
+		else
+			DeleteFile(tmpFile);
+		}
+	
+	// Something went wrong. Try DeleteFile(), even if it can invalidate
+	// open file handles.
+	if(DeleteFile(file))
+		return 0;
+
+	return -1;
+}
+
+/* 
+	A wrapper for Posix open(). The reason it is there is the FILE_SHARE_DELETE 
+	flag used in CreateFile, which allows for  posixly-correct unlink
+	(that works with open files)
+*/
+static int winOpen(const char *filename, int flags,...)
+{
+	DWORD access;
+	if (flags & O_WRONLY)
+		access = GENERIC_WRITE;
+	else if (flags & O_RDWR)
+		access = GENERIC_READ|GENERIC_WRITE;
+	else
+		access = GENERIC_READ;
+
+	DWORD disposition;
+	if (flags & O_CREAT)
+		disposition = CREATE_NEW;
+	else
+		disposition = OPEN_EXISTING;
+
+	DWORD attributes;
+	if (flags & O_RANDOM)
+		attributes = FILE_FLAG_RANDOM_ACCESS;
+	else
+		attributes = FILE_ATTRIBUTE_NORMAL;
+
+	HANDLE hFile = CreateFile(filename, access, 
+		FILE_SHARE_DELETE, NULL, disposition, attributes, NULL);
+	
+	if (hFile == INVALID_HANDLE_VALUE)
+		{
+		switch(GetLastError())
+			{
+			case ERROR_ACCESS_DENIED:
+				errno = EACCES;
+				break;
+			case ERROR_FILE_NOT_FOUND:
+			case ERROR_PATH_NOT_FOUND:
+				errno = ENOENT;
+				break;
+			default:
+				errno = EINVAL;
+				break;
+			}
+		return -1;
+		}
+	return _open_osfhandle((intptr_t)hFile, 
+		flags & (_O_APPEND|_O_RDONLY|_O_TEXT));
+}
+
+#endif
+
 void IO::deleteFile(const char* fileName)
 {
-	JString path = getPath(fileName);
-	unlink(path.getString());
+	if(fileName && *fileName)
+		{
+		JString path = getPath(fileName);
+		POSIX_UNLINK_FILE(path.getString());
+		}
 }
 
 void IO::tracePage(Bdb* bdb)
@@ -810,3 +1020,4 @@ uint16 IO::computeChecksum(Page *page, s
 	return (uint16) sum;
 
 }
+

=== modified file 'storage/falcon/IOx.h'
--- a/storage/falcon/IOx.h	2008-09-11 10:56:00 +0000
+++ b/storage/falcon/IOx.h	2008-10-21 20:05:54 +0000
@@ -52,6 +52,7 @@ public:
 	bool	trialRead (Bdb *bdb);
 	void	deleteFile();
 	void	writeHeader (Hdr *header);
+	static int	minTableSpaceSize();
 	int		read(int length, UCHAR *buffer);
 	void	write(uint32 length, const UCHAR *data);
 	static bool	doesFileExist(const char *fileName);
@@ -67,6 +68,8 @@ public:
 	static	void setBaseDirectory(const char *path);
 	static	void setWriteFlags(int fileId, bool *forceFsync);
 	bool	openFile (const char *name, bool readOnly);
+	static	bool isSameFile (const char *filename1, const char *filename2);
+	void	dupFile(IO *otherFile);
 	void	longSeek(int64 offset);
 	void	read(int64 offset, int length, UCHAR* buffer);
 	void	write(int64 offset, int length, const UCHAR* buffer);

=== modified file 'storage/falcon/SRLCreateTableSpace.cpp'
--- a/storage/falcon/SRLCreateTableSpace.cpp	2008-07-17 13:52:17 +0000
+++ b/storage/falcon/SRLCreateTableSpace.cpp	2008-10-21 20:05:54 +0000
@@ -87,7 +87,11 @@ void SRLCreateTableSpace::pass1()
 {
 	TableSpaceInit tsInit;
 	tsInit.comment		= comment;
-	log->database->tableSpaceManager->redoCreateTableSpace(tableSpaceId, nameLength, name, filenameLength, filename, type, &tsInit);
+	log->database->tableSpaceManager->redoCreateTableSpace(tableSpaceId,
+		nameLength, name, filenameLength, filename, type, &tsInit);
+	TableSpace *tablespace = log->tableSpaceManager->findTableSpace(tableSpaceId);
+	log->setDataFileActive(tablespace->filename);
+
 }
 
 void SRLCreateTableSpace::pass2()

=== modified file 'storage/falcon/SRLDropTableSpace.cpp'
--- a/storage/falcon/SRLDropTableSpace.cpp	2008-06-08 22:12:35 +0000
+++ b/storage/falcon/SRLDropTableSpace.cpp	2008-10-21 20:05:54 +0000
@@ -65,7 +65,8 @@ void SRLDropTableSpace::read()
 
 void SRLDropTableSpace::pass1()
 {
-
+	TableSpace *tablespace = log->tableSpaceManager->findTableSpace(tableSpaceId);
+	log->setDataFileInactive(tablespace->filename);
 }
 
 void SRLDropTableSpace::pass2()
@@ -80,5 +81,11 @@ void SRLDropTableSpace::commit()
 
 void SRLDropTableSpace::redo()
 {
-	log->tableSpaceManager->expungeTableSpace(tableSpaceId);
+
+	TableSpace *tablespace = log->tableSpaceManager->findTableSpace(tableSpaceId);
+	if(!tablespace)
+		return;
+	bool removeFile = !log->isDataFileActive(tablespace->filename);
+	log->tableSpaceManager->expungeTableSpace(tableSpaceId, removeFile);
+
 }

=== modified file 'storage/falcon/SerialLog.cpp'
--- a/storage/falcon/SerialLog.cpp	2008-10-20 21:28:11 +0000
+++ b/storage/falcon/SerialLog.cpp	2008-10-21 20:05:54 +0000
@@ -147,6 +147,8 @@ SerialLog::~SerialLog()
 	delete logControl;
 	delete recoveryIndexes;
 	delete recoverySections;
+	cleanupRecoveryDatafiles();
+
 	SerialLogWindow *window;
 
 	while ( (window = firstWindow) )
@@ -323,6 +325,9 @@ void SerialLog::recover()
 	recoveryPages = new RecoveryObjects(this);
 	recoverySections = new RecoveryObjects(this);
 	recoveryIndexes = new RecoveryObjects(this);
+	recoveryDatafiles = NULL;
+	recoveryDatafilesCount = 0;
+
 	recoveryPhase = 1;	// Take Inventory (serialLogTransactions, recoveryObject states, last checkpoint)
 	
 	// Make a first pass finding records, transactions, etc.
@@ -395,9 +400,12 @@ void SerialLog::recover()
 	delete recoveryPages;
 	delete recoverySections;
 	delete recoveryIndexes;
+
 	recoveryPages = NULL;
 	recoveryIndexes = NULL;
 	recoverySections = NULL;
+	recoveryDatafiles = NULL;
+	cleanupRecoveryDatafiles();
 	
 	for (window = firstWindow; window; window = window->next)
 		if (!(window->inUse == 0 || window == writeWindow))
@@ -1274,6 +1282,78 @@ void SerialLog::setIndexActive(int id, i
 		recoveryIndexes->deleteObject(id, tableSpaceId);
 }
 
+void SerialLog::setDataFileActive(const char *filename)
+{
+	ASSERT(recovering);
+	int pos = -1;
+	for(int i=0; i< recoveryDatafilesCount;i++)
+		if(recoveryDatafiles[i] == NULL)
+			{
+			pos = i;
+			break;
+			}
+	if(pos == -1)
+		{
+		// No free slot found, reallocate array
+		pos = recoveryDatafilesCount;
+		char **newRecoveryDatafiles = new char* [recoveryDatafilesCount+1];
+
+		if(recoveryDatafiles)
+			{
+			memcpy(newRecoveryDatafiles, recoveryDatafiles, sizeof(char*)*recoveryDatafilesCount);
+			delete recoveryDatafiles;
+			}
+		recoveryDatafiles = newRecoveryDatafiles;
+		recoveryDatafilesCount++;
+		}
+	recoveryDatafiles[pos] = new char[strlen(filename)+1];
+	strcpy(recoveryDatafiles[pos], filename);
+
+}
+
+void SerialLog::setDataFileInactive(const char *filename)
+{
+	ASSERT(recovering);
+	for(int i=0; i< recoveryDatafilesCount; i++)
+		{
+		char *fname = recoveryDatafiles[i];
+
+		if(fname && IO::isSameFile(fname,filename))
+			{
+			delete(fname);
+			recoveryDatafiles[i] = 0;
+			return;
+			}
+		}
+	
+}
+
+bool SerialLog::isDataFileActive(const char *filename)
+{
+	ASSERT(recovering);
+	for(int i=0; i< recoveryDatafilesCount; i++)
+		{
+		char *fname = recoveryDatafiles[i];
+		if(fname && IO::isSameFile(fname,filename))
+			return true;
+		}
+	return false;
+}
+
+void SerialLog::cleanupRecoveryDatafiles()
+{
+	if(recoveryDatafiles)
+		{
+		for(int i=0; i< recoveryDatafilesCount;i++)
+			if(recoveryDatafiles[i])
+				delete(recoveryDatafiles[i]);
+	
+		delete(recoveryDatafiles);
+		recoveryDatafiles = NULL;
+		recoveryDatafilesCount = 0;
+		}
+}
+
 void SerialLog::setIndexInactive(int id, int tableSpaceId)
 {
 	if (!recoveryIndexes)

=== modified file 'storage/falcon/SerialLog.h'
--- a/storage/falcon/SerialLog.h	2008-10-20 21:28:11 +0000
+++ b/storage/falcon/SerialLog.h	2008-10-21 20:05:54 +0000
@@ -126,13 +126,16 @@ public:
 	void			setSectionInactive(int id, int tableSpaceId);
 	void			setIndexActive(int id, int tableSpaceId);
 	void			setIndexInactive(int id, int tableSpaceId);
+	bool			isDataFileActive(const  char *filename);
+	void			setDataFileActive(const char *filename);
+	void			setDataFileInactive(const char *filename);
+	void			cleanupRecoveryDatafiles();
 	void			updateSectionUseVector(uint sectionId, int tableSpaceId, int delta);
 	void			updateIndexUseVector(uint indexId, int tableSpaceId, int delta);
 	bool			sectionInUse(int sectionId, int tableSpaceId);
 	bool			bumpIndexIncarnation(int indexId, int tableSpaceId, int state);
 	bool			bumpSectionIncarnation (int sectionId, int tableSpaceId, int state);
 	bool			bumpPageIncarnation (int32 pageNumber, int tableSpaceId, int state);
-
 	void			redoFreePage(int32 pageNumber, int tableSpaceId);
 	
 	bool			indexInUse(int indexId, int tableSpaceId);
@@ -170,6 +173,8 @@ public:
 	RecoveryObjects		*recoveryPages;
 	RecoveryObjects		*recoverySections;
 	RecoveryObjects		*recoveryIndexes;
+	char**				recoveryDatafiles;
+	int					recoveryDatafilesCount;
 	Dbb					*defaultDbb;
 	Gopher				*gophers;
 	Thread				*srlQueue;

=== modified file 'storage/falcon/SerialLogFile.cpp'
--- a/storage/falcon/SerialLogFile.cpp	2008-10-16 02:53:35 +0000
+++ b/storage/falcon/SerialLogFile.cpp	2008-10-21 20:05:54 +0000
@@ -369,8 +369,8 @@ void SerialLogFile::zap()
 	size_t initialSize = MAX(sectorSize, 8192); 
 	UCHAR *junk = new UCHAR[initialSize +sectorSize];
 	UCHAR *buffer = ALIGN(junk, sectorSize);
-	memset(buffer, 0, sectorSize);
-	write(0, sectorSize, (SerialLogBlock*) buffer);
+	memset(buffer, 0, initialSize);
+	write(0, initialSize, (SerialLogBlock*) buffer);
 	delete junk;
 }
 

=== modified file 'storage/falcon/StorageHandler.cpp'
--- a/storage/falcon/StorageHandler.cpp	2008-10-01 03:13:44 +0000
+++ b/storage/falcon/StorageHandler.cpp	2008-10-21 20:05:54 +0000
@@ -508,9 +508,6 @@ int StorageHandler::createTablespace(con
 	TableSpaceManager *tableSpaceManager =
 		dictionaryConnection->database->tableSpaceManager;
 
-	if (!tableSpaceManager->waitForPendingDrop(filename, 10))
-		// file still exists after waiting for 10 seconds
-		return  StorageErrorTableSpaceDataFileExist;
 
 	try
 		{

=== modified file 'storage/falcon/TableSpace.cpp'
--- a/storage/falcon/TableSpace.cpp	2008-08-18 20:17:15 +0000
+++ b/storage/falcon/TableSpace.cpp	2008-10-21 20:05:54 +0000
@@ -59,21 +59,23 @@ TableSpace::~TableSpace()
 	delete dbb;
 }
 
-void TableSpace::open()
+void TableSpace::open(IO *file)
 {
 	try
 		{
-		dbb->openFile(filename, false);
+		if(!file)
+			dbb->openFile(filename, false);
+		else
+			{
+			dbb->dupFile(file);
+			dbb->longSeek(0);
+			}
+
 		active = true;
 		}
 	catch (SQLException&)
 		{
-		//if (dbb->doesFileExits(fileName))
 			throw;
-
-		create();
-
-		return;
 		}
 
 	Hdr	header;

=== modified file 'storage/falcon/TableSpace.h'
--- a/storage/falcon/TableSpace.h	2008-07-17 13:52:17 +0000
+++ b/storage/falcon/TableSpace.h	2008-10-21 20:05:54 +0000
@@ -36,6 +36,7 @@ struct TableSpaceInit
 class Dbb;
 class Database;
 class InfoTable;
+class IO;
 
 class TableSpace  
 {
@@ -44,7 +45,7 @@ public:
 	virtual ~TableSpace();
 
 	void	create();
-	void	open();
+	void	open(IO *file = NULL);
 	void	close(void);
 	void	shutdown(TransId transId);
 	void	dropTableSpace(void);

=== modified file 'storage/falcon/TableSpaceManager.cpp'
--- a/storage/falcon/TableSpaceManager.cpp	2008-09-22 09:24:39 +0000
+++ b/storage/falcon/TableSpaceManager.cpp	2008-10-21 20:05:54 +0000
@@ -19,6 +19,7 @@
 //////////////////////////////////////////////////////////////////////
 
 #include <memory.h>
+#include <sys/stat.h>
 #include "Engine.h"
 #include "TableSpaceManager.h"
 #include "TableSpace.h"
@@ -42,6 +43,7 @@
 #include "Log.h"
 #include "InfoTable.h"
 #include "Thread.h"
+#include "IOx.h"
 
 #ifdef _DEBUG
 #undef THIS_FILE
@@ -58,7 +60,6 @@ TableSpaceManager::TableSpaceManager(Dat
 	memset(nameHash, 0, sizeof(nameHash));
 	memset(idHash, 0, sizeof(nameHash));
 	tableSpaces = NULL;
-	pendingDrops = 0;
 	syncObject.setName("TableSpaceManager::syncObject");
 }
 
@@ -209,12 +210,7 @@ void TableSpaceManager::bootstrap(int se
 		p = EncodedDataStream::decode(p, &id, true);
 		p = EncodedDataStream::decode(p, &fileName, true);
 		p = EncodedDataStream::decode(p, &type, true);
-		/***
-		p = EncodedDataStream::decode(p, &comment, true);
 
-		TableSpaceInit tsInit;
-		tsInit.comment		= comment.getString();
-		***/
 		
 		TableSpace *tableSpace = new TableSpace(database, name.getString(), id.getInt(), fileName.getString(), type.getInt(), NULL);
 		Log::debug("New table space %s, id %d, type %d, filename %s\n", (const char*) tableSpace->name, tableSpace->tableSpaceId, tableSpace->type, (const char*) tableSpace->filename);
@@ -222,7 +218,10 @@ void TableSpaceManager::bootstrap(int se
 		if (tableSpace->type == TABLESPACE_TYPE_TABLESPACE)
 			try
 				{
-				tableSpace->open();
+				if(!IO::doesFileExist(fileName.getString()))
+					tableSpace->create();
+				else
+					tableSpace->open();
 				}
 			catch(SQLException& exception)
 				{
@@ -252,7 +251,6 @@ TableSpace* TableSpaceManager::getTableS
 		throw SQLError(COMPILE_ERROR, "can't find table space %d", id);
 
 	if (!tableSpace->active)
-		//throw SQLError(RUNTIME_ERROR, "table space \"%s\" is not active", (const char*) tableSpace->name);
 		tableSpace->open();
 
 	return tableSpace;
@@ -298,13 +296,13 @@ void TableSpaceManager::dropTableSpace(T
 			break;
 			}
 			
-	pendingDrops++;
 	syncObj.unlock();
-
+	tableSpace->active = false;
+	JString filename = tableSpace->dbb->fileName;
 	database->serialLog->logControl->dropTableSpace.append(tableSpace, transaction);
 	database->commitSystemTransaction();
+	IO::deleteFile(filename);
 
-	tableSpace->active = false;
 }
 
 void TableSpaceManager::reportStatistics(void)
@@ -334,7 +332,7 @@ void TableSpaceManager::sync()
 		tableSpace->sync();
 }
 
-void TableSpaceManager::expungeTableSpace(int tableSpaceId)
+void TableSpaceManager::expungeTableSpace(int tableSpaceId, bool removeFile)
 {
 	Sync sync(&syncObject, "TableSpaceManager::expungeTableSpace");
 	sync.lock(Exclusive);
@@ -370,12 +368,13 @@ void TableSpaceManager::expungeTableSpac
 			}
 
 	sync.unlock();
-	tableSpace->dropTableSpace();
+	if (removeFile)
+		tableSpace->dropTableSpace();
+	else
+		tableSpace->close();
+
 	delete tableSpace;
 
-	sync.lock(Exclusive);
-	if(pendingDrops >0)
-		pendingDrops--;
 }
 
 void TableSpaceManager::reportWrites(void)
@@ -406,16 +405,52 @@ void TableSpaceManager::redoCreateTableS
 	file[fileNameLength] = 0;
 	tableSpace = new TableSpace(database, buffer, id, file, type, tsInit);
 	tableSpace->needSave = true;
-	add(tableSpace);	
+
+	// Check if another tablespace already has this file open
+	// if so duplicate the file descriptor (open will fail due to file locking)
+	IO *dupIO = NULL ;
+	for(TableSpace *ts = tableSpaces; ts; ts = ts->next)
+		{
+		if (IO::isSameFile(ts->filename,file) && ts->dbb->fileId != -1)
+			dupIO = ts->dbb;
+		}
+	add(tableSpace);
 
 	try
 		{
-		tableSpace->open();
+		if(!IO::doesFileExist(file))
+			{
+			tableSpace->create();
+			return;
+			}
+
+		try
+			{
+			tableSpace->open(dupIO);
+			}
+		catch(SQLException &ex)
+			{
+			// Check if database died during tablespace creation and
+			// file has been created but header was not completely written.
+			// If it was the case, recreate the tablespace.
+			struct stat stats;
+			if (ex.getSqlcode() == IO_ERROR
+				&& IO::fileStat(file, &stats) == 0 
+				&& stats.st_size < IO::minTableSpaceSize())
+				{
+				IO::deleteFile(file);
+				tableSpace->create();
+				}
+			else
+				throw;
+			}
+
 		}
 	catch(SQLException& exception)
 		{
-		Log::log("Couldn't open table space file \"%s\" for tablespace \"%s\": %s\n", 
-					file, buffer, exception.getText());
+		Log::log("Could neither open nor create table space file \"%s\" for tablespace \"%s\": %s\n", 
+			file, buffer, exception.getText());
+		throw;
 		}
 }
 
@@ -565,26 +600,3 @@ void TableSpaceManager::getTableSpaceFil
 		}
 }
 
-
-// Wait for specified amount of time for a  file to be deleted.
-// Don't wait if pendingDrops count is 0.
-//
-// The function returns true, if wait was successfull, i.e file does not exist
-//(anymore)
-bool TableSpaceManager::waitForPendingDrop(const char  *filename, int seconds)
-{
-	bool fileExists;
-
-	do
-		{
-		fileExists = IO::doesFileExist(filename);
-		if (fileExists && pendingDrops > 0 && seconds-- > 0)
-			Thread::getThread("TransactionManager::waitForPendingDrop")->sleep(1000);
-		else
-			break;
-		}
-	while(true);
-
-	return !fileExists;
-}
-

=== modified file 'storage/falcon/TableSpaceManager.h'
--- a/storage/falcon/TableSpaceManager.h	2008-09-22 09:24:39 +0000
+++ b/storage/falcon/TableSpaceManager.h	2008-10-21 20:05:54 +0000
@@ -59,11 +59,10 @@ public:
 	void			getTableSpaceFilesInfo(InfoTable* infoTable);
 	void			validate(int optionMask);
 	void			sync();
-	void			expungeTableSpace(int tableSpaceId);
+	void			expungeTableSpace(int tableSpaceId, bool removeFile=true);
 	void			reportWrites(void);
 	void			redoCreateTableSpace(int id, int nameLength, const char* name, int fileNameLength, const char* fileName, int type, TableSpaceInit* tsInit);
 	void			initialize(void);
-	bool			waitForPendingDrop(const char *filename, int seconds);
 
 	Database	*database;
 	TableSpace	*tableSpaces;
@@ -71,7 +70,6 @@ public:
 	TableSpace	*idHash[TS_HASH_SIZE];
 	SyncObject	syncObject;
 	void postRecovery(void);
-	int pendingDrops;
 };
 
 #endif // !defined(AFX_TABLESPACEMANAGER_H__BD1D39F6_2201_4136_899C_7CB106E99B8C__INCLUDED_)