| List: | Commits | « Previous MessageNext Message » | |
| From: | tomas | Date: | June 27 2006 10:03am |
| Subject: | bk commit into 5.0 tree (tomas:1.2199) BUG#19852 | ||
| View as plain text | |||
Below is the list of changes that have just been committed into a local 5.0 repository of tomas. When tomas does a push these changes will be propagated to the main repository and, within 24 hours after the push, to the public repository. For information on how to access the public repository see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html ChangeSet 1.2199 06/06/27 10:02:58 tomas@stripped +32 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - make sure to allocate just enough pages in the fragments by using the actual row count from the backup, to avoid over allocation of pages to fragments, and thus avoid the bug sql/ha_ndbcluster.cc 1.262 06/06/27 10:02:33 tomas@stripped +8 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - set min and max rows according to sql definition ndb/tools/restore/consumer_restore.cpp 1.24 06/06/27 10:02:32 tomas@stripped +10 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - set min in restore to the actual row count (this is the actual bug fix) ndb/tools/restore/Restore.hpp 1.20 06/06/27 10:02:32 tomas@stripped +15 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - add retrieval of fragment info ndb/tools/restore/Restore.cpp 1.27 06/06/27 10:02:32 tomas@stripped +57 -1 Bug #19852 Restoring backup made from cluster with full data memory fails - add retrieval of fragment info ndb/src/ndbapi/NdbDictionaryImpl.hpp 1.39 06/06/27 10:02:32 tomas@stripped +3 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - min and max rows in dict ndb/src/ndbapi/NdbDictionaryImpl.cpp 1.94 06/06/27 10:02:32 tomas@stripped +21 -1 Bug #19852 Restoring backup made from cluster with full data memory fails - min and max rows in dict ndb/src/ndbapi/NdbDictionary.cpp 1.38 06/06/27 10:02:32 tomas@stripped +24 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - min and max rows in dict ndb/src/mgmsrv/MgmtSrvr.hpp 1.46 06/06/27 10:02:31 tomas@stripped +2 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bits on bytes and records ndb/src/mgmsrv/MgmtSrvr.cpp 1.102 06/06/27 10:02:30 tomas@stripped +8 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bits on bytes and records ndb/src/kernel/blocks/dbtup/DbtupPageMap.cpp 1.2 06/06/27 10:02:30 tomas@stripped +4 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - grow page allocation starting from 2 irrespective of first page allocation ndb/src/kernel/blocks/dbtup/DbtupMeta.cpp 1.14 06/06/27 10:02:30 tomas@stripped +51 -32 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to frag req - move memory allocation to fragment to after adding of attributes to get correct headsize - allocate pages to fragments according to min rows setting ndb/src/kernel/blocks/dbtup/Dbtup.hpp 1.26 06/06/27 10:02:30 tomas@stripped +4 -1 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to frag req ndb/src/kernel/blocks/dblqh/DblqhMain.cpp 1.88 06/06/27 10:02:29 tomas@stripped +59 -35 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to frag req ndb/src/kernel/blocks/dblqh/Dblqh.hpp 1.38 06/06/27 10:02:28 tomas@stripped +7 -3 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to frag req ndb/src/kernel/blocks/dbdict/Dbdict.hpp 1.17 06/06/27 10:02:28 tomas@stripped +4 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - added max and min rows to dict table object ndb/src/kernel/blocks/dbdict/Dbdict.cpp 1.62 06/06/27 10:02:28 tomas@stripped +29 -3 Bug #19852 Restoring backup made from cluster with full data memory fails - added max and min rows to dict table object ndb/src/kernel/blocks/backup/BackupInit.cpp 1.16 06/06/27 10:02:28 tomas@stripped +3 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - new signal fragment complete to all participants ndb/src/kernel/blocks/backup/BackupFormat.hpp 1.2 06/06/27 10:02:28 tomas@stripped +16 -1 Bug #19852 Restoring backup made from cluster with full data memory fails - new section in backup with per fragment info in ctl file - 32 -> 64 bit on bytes and records ndb/src/kernel/blocks/backup/Backup.hpp 1.13 06/06/27 10:02:28 tomas@stripped +10 -5 Bug #19852 Restoring backup made from cluster with full data memory fails - new section in backup with per fragment info in ctl file - 32 -> 64 bit on bytes and records ndb/src/kernel/blocks/backup/Backup.cpp 1.34 06/06/27 10:02:27 tomas@stripped +138 -25 Bug #19852 Restoring backup made from cluster with full data memory fails - new section in backup with per fragment info in ctl file - 32 -> 64 bit on bytes and records ndb/src/common/debugger/signaldata/LqhFrag.cpp 1.2 06/06/27 10:02:27 tomas@stripped +4 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to frag req ndb/src/common/debugger/signaldata/DictTabInfo.cpp 1.7 06/06/27 10:02:27 tomas@stripped +8 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to dict tab info ndb/src/common/debugger/signaldata/BackupSignalData.cpp 1.2 06/06/27 10:02:27 tomas@stripped +3 -3 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bit on bytes and records ndb/src/common/debugger/signaldata/BackupImpl.cpp 1.4 06/06/27 10:02:26 tomas@stripped +4 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bit on bytes and records ndb/include/ndbapi/NdbDictionary.hpp 1.55 06/06/27 10:02:26 tomas@stripped +14 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - added get/set of min max rows ndb/include/kernel/signaldata/TupFrag.hpp 1.9 06/06/27 10:02:25 tomas@stripped +13 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to add frag req ndb/include/kernel/signaldata/LqhFrag.hpp 1.3 06/06/27 10:02:25 tomas@stripped +15 -10 Bug #19852 Restoring backup made from cluster with full data memory fails - added min and max rows to add frag req ndb/include/kernel/signaldata/DictTabInfo.hpp 1.19 06/06/27 10:02:24 tomas@stripped +11 -0 Bug #19852 Restoring backup made from cluster with full data memory fails - add min and max rows to dict tab info ndb/include/kernel/signaldata/BackupSignalData.hpp 1.5 06/06/27 10:02:24 tomas@stripped +5 -3 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bit on bytes and records ndb/include/kernel/signaldata/BackupImpl.hpp 1.5 06/06/27 10:02:24 tomas@stripped +19 -3 Bug #19852 Restoring backup made from cluster with full data memory fails - 32 -> 64 bit on bytes and records - new signal fragment complete to all participants ndb/include/kernel/signaldata/BackupContinueB.hpp 1.2 06/06/27 10:02:24 tomas@stripped +2 -1 Bug #19852 Restoring backup made from cluster with full data memory fails - time slica writing of fragment info to ctl file ndb/include/kernel/GlobalSignalNumbers.h 1.11 06/06/27 10:02:24 tomas@stripped +2 -2 Bug #19852 Restoring backup made from cluster with full data memory fails - distribute fragment complete to all participants to update row count # This is a BitKeeper patch. What follows are the unified diffs for the # set of deltas contained in the patch. The rest of the patch, the part # that BitKeeper cares about, is below these diffs. # User: tomas # Host: poseidon.ndb.mysql.com # Root: /home/tomas/mysql-5.0-main --- 1.10/ndb/include/kernel/GlobalSignalNumbers.h 2005-09-20 09:34:43 +02:00 +++ 1.11/ndb/include/kernel/GlobalSignalNumbers.h 2006-06-27 10:02:24 +02:00 @@ -611,8 +611,6 @@ #define GSN_WAIT_GCP_REF 500 #define GSN_WAIT_GCP_CONF 501 -/* 502 not used */ - /** * Trigger and index signals */ @@ -681,6 +679,8 @@ #define GSN_BACKUP_FRAGMENT_REQ 545 #define GSN_BACKUP_FRAGMENT_REF 546 #define GSN_BACKUP_FRAGMENT_CONF 547 + +#define GSN_BACKUP_FRAGMENT_COMPLETE_REP 502 #define GSN_STOP_BACKUP_REQ 548 #define GSN_STOP_BACKUP_REF 549 --- 1.1/ndb/include/kernel/signaldata/BackupContinueB.hpp 2004-04-14 10:23:53 +02:00 +++ 1.2/ndb/include/kernel/signaldata/BackupContinueB.hpp 2006-06-27 10:02:24 +02:00 @@ -31,7 +31,8 @@ BUFFER_UNDERFLOW = 1, BUFFER_FULL_SCAN = 2, BUFFER_FULL_FRAG_COMPLETE = 3, - BUFFER_FULL_META = 4 + BUFFER_FULL_META = 4, + BACKUP_FRAGMENT_INFO = 5 }; }; --- 1.4/ndb/include/kernel/signaldata/BackupImpl.hpp 2005-09-08 11:34:51 +02:00 +++ 1.5/ndb/include/kernel/signaldata/BackupImpl.hpp 2006-06-27 10:02:24 +02:00 @@ -258,15 +258,31 @@ friend bool printBACKUP_FRAGMENT_CONF(FILE *, const Uint32 *, Uint32, Uint16); public: - STATIC_CONST( SignalLength = 6 ); + STATIC_CONST( SignalLength = 8 ); private: Uint32 backupId; Uint32 backupPtr; Uint32 tableId; Uint32 fragmentNo; - Uint32 noOfRecords; - Uint32 noOfBytes; + Uint32 noOfRecordsLow; + Uint32 noOfBytesLow; + Uint32 noOfRecordsHigh; + Uint32 noOfBytesHigh; +}; + +class BackupFragmentCompleteRep { +public: + STATIC_CONST( SignalLength = 8 ); + + Uint32 backupId; + Uint32 backupPtr; + Uint32 tableId; + Uint32 fragmentNo; + Uint32 noOfTableRowsLow; + Uint32 noOfFragmentRowsLow; + Uint32 noOfTableRowsHigh; + Uint32 noOfFragmentRowsHigh; }; class StopBackupReq { --- 1.4/ndb/include/kernel/signaldata/BackupSignalData.hpp 2005-09-08 11:34:51 +02:00 +++ 1.5/ndb/include/kernel/signaldata/BackupSignalData.hpp 2006-06-27 10:02:24 +02:00 @@ -201,17 +201,19 @@ friend bool printBACKUP_COMPLETE_REP(FILE *, const Uint32 *, Uint32, Uint16); public: - STATIC_CONST( SignalLength = 8 + NdbNodeBitmask::Size ); + STATIC_CONST( SignalLength = 10 + NdbNodeBitmask::Size ); private: Uint32 senderData; Uint32 backupId; Uint32 startGCP; Uint32 stopGCP; - Uint32 noOfBytes; - Uint32 noOfRecords; + Uint32 noOfBytesLow; + Uint32 noOfRecordsLow; Uint32 noOfLogBytes; Uint32 noOfLogRecords; NdbNodeBitmask nodes; + Uint32 noOfBytesHigh; + Uint32 noOfRecordsHigh; }; /** --- 1.18/ndb/include/kernel/signaldata/DictTabInfo.hpp 2005-07-20 13:22:36 +02:00 +++ 1.19/ndb/include/kernel/signaldata/DictTabInfo.hpp 2006-06-27 10:02:24 +02:00 @@ -117,9 +117,16 @@ CustomTriggerId = 25, FrmLen = 26, FrmData = 27, + FragmentCount = 128, // No of fragments in table (!fragment replicas) FragmentDataLen = 129, FragmentData = 130, // CREATE_FRAGMENTATION reply + + MaxRowsLow = 139, + MaxRowsHigh = 140, + MinRowsLow = 133, + MinRowsHigh = 144, + TableEnd = 999, AttributeName = 1000, // String, Mandatory @@ -263,6 +270,10 @@ Uint32 FragmentCount; Uint32 FragmentDataLen; Uint16 FragmentData[(MAX_FRAGMENT_DATA_BYTES+1)/2]; + Uint32 MaxRowsLow; + Uint32 MaxRowsHigh; + Uint32 MinRowsLow; + Uint32 MinRowsHigh; void init(); }; --- 1.2/ndb/include/kernel/signaldata/LqhFrag.hpp 2004-09-16 12:53:16 +02:00 +++ 1.3/ndb/include/kernel/signaldata/LqhFrag.hpp 2006-06-27 10:02:25 +02:00 @@ -104,7 +104,7 @@ friend bool printLQH_FRAG_REQ(FILE *, const Uint32 *, Uint32, Uint16); public: - STATIC_CONST( SignalLength = 25 ); + STATIC_CONST( SignalLength = 24 ); enum RequestInfo { CreateInRunning = 0x8000000, @@ -115,27 +115,32 @@ Uint32 senderData; Uint32 senderRef; Uint32 fragmentId; - Uint32 requestInfo; + Uint8 requestInfo; + Uint8 unused1; + Uint16 noOfAttributes; Uint32 tableId; Uint32 localKeyLength; - Uint32 maxLoadFactor; - Uint32 minLoadFactor; - Uint32 kValue; + Uint16 maxLoadFactor; + Uint16 minLoadFactor; + Uint16 kValue; + Uint8 tableType; // DictTabInfo::TableType + Uint8 GCPIndicator; Uint32 lh3DistrBits; Uint32 lh3PageBits; - Uint32 noOfAttributes; Uint32 noOfNullAttributes; - Uint32 noOfPagesToPreAllocate; + Uint32 maxRowsLow; + Uint32 maxRowsHigh; + Uint32 minRowsLow; + Uint32 minRowsHigh; Uint32 schemaVersion; Uint32 keyLength; Uint32 nextLCP; Uint32 noOfKeyAttr; - Uint32 noOfNewAttr; // noOfCharsets in upper half + Uint16 noOfNewAttr; + Uint16 noOfCharsets; Uint32 checksumIndicator; Uint32 noOfAttributeGroups; - Uint32 GCPIndicator; Uint32 startGci; - Uint32 tableType; // DictTabInfo::TableType Uint32 primaryTableId; // table of index or RNIL }; --- 1.8/ndb/include/kernel/signaldata/TupFrag.hpp 2006-04-20 12:04:28 +02:00 +++ 1.9/ndb/include/kernel/signaldata/TupFrag.hpp 2006-06-27 10:02:25 +02:00 @@ -30,7 +30,7 @@ friend class Dblqh; friend class Dbtup; public: - STATIC_CONST( SignalLength = 14 ); + STATIC_CONST( SignalLength = 17 ); private: Uint32 userPtr; Uint32 userRef; @@ -38,7 +38,18 @@ Uint32 tableId; Uint32 noOfAttr; Uint32 fragId; - Uint32 todo[8]; + Uint32 maxRowsLow; + Uint32 maxRowsHigh; + Uint32 minRowsLow; + Uint32 minRowsHigh; + Uint32 noOfNullAttr; + Uint32 schemaVersion; + Uint32 noOfKeyAttr; + Uint16 noOfNewAttr; + Uint16 noOfCharsets; + Uint32 checksumIndicator; + Uint32 noOfAttributeGroups; + Uint32 globalCheckpointIdIndicator; }; class TupFragConf { --- 1.54/ndb/include/ndbapi/NdbDictionary.hpp 2006-05-24 10:57:27 +02:00 +++ 1.55/ndb/include/ndbapi/NdbDictionary.hpp 2006-06-27 10:02:26 +02:00 @@ -722,6 +722,20 @@ */ void setObjectType(Object::Type type); + /** + * Set/Get Maximum number of rows in table (only used to calculate + * number of partitions). + */ + void setMaxRows(Uint64 maxRows); + Uint64 getMaxRows() const; + + /** + * Set/Get Minimum number of rows in table (only used to calculate + * number of partitions). + */ + void setMinRows(Uint64 minRows); + Uint64 getMinRows() const; + /** @} *******************************************************************/ #ifndef DOXYGEN_SHOULD_SKIP_INTERNAL --- 1.3/ndb/src/common/debugger/signaldata/BackupImpl.cpp 2005-04-22 09:07:22 +02:00 +++ 1.4/ndb/src/common/debugger/signaldata/BackupImpl.cpp 2006-06-27 10:02:26 +02:00 @@ -100,8 +100,10 @@ BackupFragmentConf* sig = (BackupFragmentConf*)data; fprintf(out, " backupPtr: %d backupId: %d\n", sig->backupPtr, sig->backupId); - fprintf(out, " tableId: %d fragmentNo: %d records: %d bytes: %d\n", - sig->tableId, sig->fragmentNo, sig->noOfRecords, sig->noOfBytes); + fprintf(out, " tableId: %d fragmentNo: %d records: %llu bytes: %llu\n", + sig->tableId, sig->fragmentNo, + sig->noOfRecordsLow + (((Uint64)sig->noOfRecordsHigh) << 32), + sig->noOfBytesLow + (((Uint64)sig->noOfBytesHigh) << 32)); return true; } --- 1.1/ndb/src/common/debugger/signaldata/BackupSignalData.cpp 2004-04-14 10:24:07 +02:00 +++ 1.2/ndb/src/common/debugger/signaldata/BackupSignalData.cpp 2006-06-27 10:02:27 +02:00 @@ -72,11 +72,11 @@ bool printBACKUP_COMPLETE_REP(FILE * out, const Uint32 * data, Uint32 len, Uint16 b){ BackupCompleteRep* sig = (BackupCompleteRep*)data; - fprintf(out, " senderData: %d backupId: %d records: %d bytes: %d\n", + fprintf(out, " senderData: %d backupId: %d records: %llu bytes: %llu\n", sig->senderData, sig->backupId, - sig->noOfRecords, - sig->noOfBytes); + sig->noOfRecordsLow + (((Uint64)sig->noOfRecordsHigh) << 32), + sig->noOfBytesLow + (((Uint64)sig->noOfBytesHigh) << 32)); return true; } --- 1.6/ndb/src/common/debugger/signaldata/DictTabInfo.cpp 2005-01-10 14:40:51 +01:00 +++ 1.7/ndb/src/common/debugger/signaldata/DictTabInfo.cpp 2006-06-27 10:02:27 +02:00 @@ -48,6 +48,10 @@ DTIMAP(Table, FragmentCount, FragmentCount), DTIMAP2(Table, FragmentDataLen, FragmentDataLen, 0, MAX_FRAGMENT_DATA_BYTES), DTIMAPB(Table, FragmentData, FragmentData, 0, MAX_FRAGMENT_DATA_BYTES, FragmentDataLen), + DTIMAP(Table, MaxRowsLow, MaxRowsLow), + DTIMAP(Table, MaxRowsHigh, MaxRowsHigh), + DTIMAP(Table, MinRowsLow, MinRowsLow), + DTIMAP(Table, MinRowsHigh, MinRowsHigh), DTIBREAK(AttributeName) }; @@ -124,6 +128,10 @@ FragmentCount = 0; FragmentDataLen = 0; memset(FragmentData, 0, sizeof(FragmentData)); + MaxRowsLow = 0; + MaxRowsHigh = 0; + MinRowsLow = 0; + MinRowsHigh = 0; } void --- 1.1/ndb/src/common/debugger/signaldata/LqhFrag.cpp 2004-04-14 10:24:08 +02:00 +++ 1.2/ndb/src/common/debugger/signaldata/LqhFrag.cpp 2006-06-27 10:02:27 +02:00 @@ -37,8 +37,10 @@ fprintf(output, " noOfAttributes: %d noOfNullAttributes: %d keyLength: %d\n", sig->noOfAttributes, sig->noOfNullAttributes, sig->keyLength); - fprintf(output, " noOfPagesToPreAllocate: %d schemaVersion: %d nextLCP: %d\n", - sig->noOfPagesToPreAllocate, sig->schemaVersion, sig->nextLCP); + fprintf(output, " maxRowsLow/High: %u/%u minRowsLow/High: %u/%u\n", + sig->maxRowsLow, sig->maxRowsHigh, sig->minRowsLow, sig->minRowsHigh); + fprintf(output, " schemaVersion: %d nextLCP: %d\n", + sig->schemaVersion, sig->nextLCP); return true; } --- 1.33/ndb/src/kernel/blocks/backup/Backup.cpp 2006-04-26 16:02:32 +02:00 +++ 1.34/ndb/src/kernel/blocks/backup/Backup.cpp 2006-06-27 10:02:27 +02:00 @@ -266,6 +266,65 @@ const Uint32 Tdata2 = signal->theData[2]; switch(Tdata0) { + case BackupContinueB::BACKUP_FRAGMENT_INFO: + { + const Uint32 ptr_I = Tdata1; + Uint32 tabPtr_I = Tdata2; + Uint32 fragPtr_I = signal->theData[3]; + + BackupRecordPtr ptr; + c_backupPool.getPtr(ptr, ptr_I); + TablePtr tabPtr; + ptr.p->tables.getPtr(tabPtr, tabPtr_I); + FragmentPtr fragPtr; + tabPtr.p->fragments.getPtr(fragPtr, fragPtr_I); + + BackupFilePtr filePtr; + ptr.p->files.getPtr(filePtr, ptr.p->ctlFilePtr); + + const Uint32 sz = sizeof(BackupFormat::CtlFile::FragmentInfo) >> 2; + Uint32 * dst; + if (!filePtr.p->operation.dataBuffer.getWritePtr(&dst, sz)) + { + sendSignalWithDelay(BACKUP_REF, GSN_CONTINUEB, signal, 100, 4); + return; + } + + BackupFormat::CtlFile::FragmentInfo * fragInfo = + (BackupFormat::CtlFile::FragmentInfo*)dst; + fragInfo->SectionType = htonl(BackupFormat::FRAGMENT_INFO); + fragInfo->SectionLength = htonl(sz); + fragInfo->TableId = htonl(fragPtr.p->tableId); + fragInfo->FragmentNo = htonl(fragPtr_I); + fragInfo->NoOfRecordsLow = htonl(fragPtr.p->noOfRecords & 0xFFFFFFFF); + fragInfo->NoOfRecordsHigh = htonl(fragPtr.p->noOfRecords >> 32); + fragInfo->FilePosLow = htonl(0 & 0xFFFFFFFF); + fragInfo->FilePosHigh = htonl(0 >> 32); + + filePtr.p->operation.dataBuffer.updateWritePtr(sz); + + fragPtr_I++; + if (fragPtr_I == tabPtr.p->fragments.getSize()) + { + signal->theData[0] = tabPtr.p->tableId; + signal->theData[1] = 0; // unlock + EXECUTE_DIRECT(DBDICT, GSN_BACKUP_FRAGMENT_REQ, signal, 2); + + fragPtr_I = 0; + ptr.p->tables.next(tabPtr); + if ((tabPtr_I = tabPtr.i) == RNIL) + { + closeFiles(signal, ptr); + return; + } + } + signal->theData[0] = BackupContinueB::BACKUP_FRAGMENT_INFO; + signal->theData[1] = ptr_I; + signal->theData[2] = tabPtr_I; + signal->theData[3] = fragPtr_I; + sendSignal(BACKUP_REF, GSN_CONTINUEB, signal, 4, JBB); + return; + } case BackupContinueB::START_FILE_THREAD: case BackupContinueB::BUFFER_UNDERFLOW: { @@ -455,7 +514,7 @@ return false; } -static Uint32 xps(Uint32 x, Uint64 ms) +static Uint32 xps(Uint64 x, Uint64 ms) { float fx = x; float fs = ms; @@ -469,9 +528,9 @@ } struct Number { - Number(Uint32 r) { val = r;} - Number & operator=(Uint32 r) { val = r; return * this; } - Uint32 val; + Number(Uint64 r) { val = r;} + Number & operator=(Uint64 r) { val = r; return * this; } + Uint64 val; }; NdbOut & @@ -545,8 +604,10 @@ startTime = NdbTick_CurrentMillisecond() - startTime; ndbout_c("Backup %d has completed", rep->backupId); - const Uint32 bytes = rep->noOfBytes; - const Uint32 records = rep->noOfRecords; + const Uint64 bytes = + rep->noOfBytesLow + (((Uint64)rep->noOfBytesHigh) << 32); + const Uint64 records = + rep->noOfRecordsLow + (((Uint64)rep->noOfRecordsHigh) << 32); Number rps = xps(records, startTime); Number bps = xps(bytes, startTime); @@ -1905,8 +1966,10 @@ const Uint32 tableId = conf->tableId; const Uint32 fragmentNo = conf->fragmentNo; const Uint32 nodeId = refToNode(signal->senderBlockRef()); - const Uint32 noOfBytes = conf->noOfBytes; - const Uint32 noOfRecords = conf->noOfRecords; + const Uint64 noOfBytes = + conf->noOfBytesLow + (((Uint64)conf->noOfBytesHigh) << 32); + const Uint64 noOfRecords = + conf->noOfRecordsLow + (((Uint64)conf->noOfRecordsHigh) << 32); BackupRecordPtr ptr; c_backupPool.getPtr(ptr, ptrI); @@ -1918,9 +1981,13 @@ TablePtr tabPtr; ndbrequire(findTable(ptr, tabPtr, tableId)); + tabPtr.p->noOfRecords += noOfRecords; + FragmentPtr fragPtr; tabPtr.p->fragments.getPtr(fragPtr, fragmentNo); + fragPtr.p->noOfRecords = noOfRecords; + ndbrequire(fragPtr.p->scanned == 0); ndbrequire(fragPtr.p->scanning == 1); ndbrequire(fragPtr.p->node == nodeId); @@ -1944,6 +2011,24 @@ } else { + NodeBitmask nodes = ptr.p->nodes; + nodes.clear(getOwnNodeId()); + if (!nodes.isclear()) + { + BackupFragmentCompleteRep *rep = + (BackupFragmentCompleteRep*)signal->getDataPtrSend(); + rep->backupId = ptr.p->backupId; + rep->backupPtr = ptr.i; + rep->tableId = tableId; + rep->fragmentNo = fragmentNo; + rep->noOfTableRowsLow = (Uint32)(tabPtr.p->noOfRecords & 0xFFFFFFFF); + rep->noOfTableRowsHigh = (Uint32)(tabPtr.p->noOfRecords >> 32); + rep->noOfFragmentRowsLow = (Uint32)(noOfRecords & 0xFFFFFFFF); + rep->noOfFragmentRowsHigh = (Uint32)(noOfRecords >> 32); + NodeReceiverGroup rg(BACKUP, ptr.p->nodes); + sendSignal(rg, GSN_BACKUP_FRAGMENT_COMPLETE_REP, signal, + BackupFragmentCompleteRep::SignalLength, JBB); + } nextFragment(signal, ptr); } } @@ -2006,6 +2091,29 @@ execABORT_BACKUP_ORD(signal); } +void +Backup::execBACKUP_FRAGMENT_COMPLETE_REP(Signal* signal) +{ + jamEntry(); + BackupFragmentCompleteRep * rep = + (BackupFragmentCompleteRep*)signal->getDataPtr(); + + BackupRecordPtr ptr; + c_backupPool.getPtr(ptr, rep->backupPtr); + + TablePtr tabPtr; + ndbrequire(findTable(ptr, tabPtr, rep->tableId)); + + tabPtr.p->noOfRecords = + rep->noOfTableRowsLow + (((Uint64)rep->noOfTableRowsHigh) << 32); + + FragmentPtr fragPtr; + tabPtr.p->fragments.getPtr(fragPtr, rep->fragmentNo); + + fragPtr.p->noOfRecords = + rep->noOfFragmentRowsLow + (((Uint64)rep->noOfFragmentRowsHigh) << 32); +} + /***************************************************************************** * * Master functionallity - Drop triggers @@ -2206,8 +2314,10 @@ rep->senderData = ptr.p->clientData; rep->startGCP = ptr.p->startGCP; rep->stopGCP = ptr.p->stopGCP; - rep->noOfBytes = ptr.p->noOfBytes; - rep->noOfRecords = ptr.p->noOfRecords; + rep->noOfBytesLow = (Uint32)(ptr.p->noOfBytes & 0xFFFFFFFF); + rep->noOfRecordsLow = (Uint32)(ptr.p->noOfRecords & 0xFFFFFFFF); + rep->noOfBytesHigh = (Uint32)(ptr.p->noOfBytes >> 32); + rep->noOfRecordsHigh = (Uint32)(ptr.p->noOfRecords >> 32); rep->noOfLogBytes = ptr.p->noOfLogBytes; rep->noOfLogRecords = ptr.p->noOfLogRecords; rep->nodes = ptr.p->nodes; @@ -2220,12 +2330,14 @@ signal->theData[2] = ptr.p->backupId; signal->theData[3] = ptr.p->startGCP; signal->theData[4] = ptr.p->stopGCP; - signal->theData[5] = ptr.p->noOfBytes; - signal->theData[6] = ptr.p->noOfRecords; + signal->theData[5] = (Uint32)(ptr.p->noOfBytes & 0xFFFFFFFF); + signal->theData[6] = (Uint32)(ptr.p->noOfRecords & 0xFFFFFFFF); signal->theData[7] = ptr.p->noOfLogBytes; signal->theData[8] = ptr.p->noOfLogRecords; ptr.p->nodes.copyto(NdbNodeBitmask::Size, signal->theData+9); - sendSignal(CMVMI_REF, GSN_EVENT_REP, signal, 9+NdbNodeBitmask::Size, JBB); + signal->theData[9+NdbNodeBitmask::Size] = (Uint32)(ptr.p->noOfBytes >> 32); + signal->theData[10+NdbNodeBitmask::Size] = (Uint32)(ptr.p->noOfRecords >> 32); + sendSignal(CMVMI_REF, GSN_EVENT_REP, signal, 11+NdbNodeBitmask::Size, JBB); } else { @@ -2988,6 +3100,7 @@ /** * Initialize table object */ + tabPtr.p->noOfRecords = 0; tabPtr.p->schemaVersion = tmpTab.TableVersion; tabPtr.p->noOfAttributes = tmpTab.NoOfAttributes; tabPtr.p->noOfNull = 0; @@ -3695,8 +3808,10 @@ conf->backupPtr = ptr.i; conf->tableId = filePtr.p->tableId; conf->fragmentNo = filePtr.p->fragmentNo; - conf->noOfRecords = op.noOfRecords; - conf->noOfBytes = op.noOfBytes; + conf->noOfRecordsLow = (Uint32)(op.noOfRecords & 0xFFFFFFFF); + conf->noOfRecordsHigh = (Uint32)(op.noOfRecords >> 32); + conf->noOfBytesLow = (Uint32)(op.noOfBytes & 0xFFFFFFFF); + conf->noOfBytesHigh = (Uint32)(op.noOfBytes >> 32); sendSignal(ptr.p->masterRef, GSN_BACKUP_FRAGMENT_CONF, signal, BackupFragmentConf::SignalLength, JBB); @@ -4123,20 +4238,18 @@ gcp->StartGCP = htonl(startGCP); gcp->StopGCP = htonl(stopGCP - 1); filePtr.p->operation.dataBuffer.updateWritePtr(gcpSz); - } - { - TablePtr tabPtr; - for(ptr.p->tables.first(tabPtr); tabPtr.i != RNIL; - ptr.p->tables.next(tabPtr)) { - signal->theData[0] = tabPtr.p->tableId; - signal->theData[1] = 0; // unlock - EXECUTE_DIRECT(DBDICT, GSN_BACKUP_FRAGMENT_REQ, signal, 2); + TablePtr tabPtr; + ptr.p->tables.first(tabPtr); + + signal->theData[0] = BackupContinueB::BACKUP_FRAGMENT_INFO; + signal->theData[1] = ptr.i; + signal->theData[2] = tabPtr.i; + signal->theData[3] = 0; + sendSignal(BACKUP_REF, GSN_CONTINUEB, signal, 4, JBB); } } - - closeFiles(signal, ptr); } void --- 1.12/ndb/src/kernel/blocks/backup/Backup.hpp 2005-10-04 20:05:52 +02:00 +++ 1.13/ndb/src/kernel/blocks/backup/Backup.hpp 2006-06-27 10:02:28 +02:00 @@ -68,6 +68,7 @@ void execBACKUP_DATA(Signal* signal); void execSTART_BACKUP_REQ(Signal* signal); void execBACKUP_FRAGMENT_REQ(Signal* signal); + void execBACKUP_FRAGMENT_COMPLETE_REP(Signal* signal); void execSTOP_BACKUP_REQ(Signal* signal); void execBACKUP_STATUS_REQ(Signal* signal); void execABORT_BACKUP_ORD(Signal* signal); @@ -183,10 +184,12 @@ typedef Ptr<Attribute> AttributePtr; struct Fragment { + Uint64 noOfRecords; Uint32 tableId; - Uint32 node; - Uint16 scanned; // 0 = not scanned x = scanned by node x - Uint16 scanning; // 0 = not scanning x = scanning on node x + Uint8 node; + Uint8 scanned; // 0 = not scanned x = scanned by node x + Uint8 scanning; // 0 = not scanning x = scanning on node x + Uint8 unused1; Uint32 nextPool; }; typedef Ptr<Fragment> FragmentPtr; @@ -194,6 +197,8 @@ struct Table { Table(ArrayPool<Attribute> &, ArrayPool<Fragment> &); + Uint64 noOfRecords; + Uint32 tableId; Uint32 schemaVersion; Uint32 tableType; @@ -269,8 +274,8 @@ Uint32 tablePtr; // Ptr.i to current table FsBuffer dataBuffer; - Uint32 noOfRecords; - Uint32 noOfBytes; + Uint64 noOfRecords; + Uint64 noOfBytes; Uint32 maxRecordSize; private: --- 1.1/ndb/src/kernel/blocks/backup/BackupFormat.hpp 2004-04-14 10:24:18 +02:00 +++ 1.2/ndb/src/kernel/blocks/backup/BackupFormat.hpp 2006-06-27 10:02:28 +02:00 @@ -32,7 +32,8 @@ FRAGMENT_FOOTER = 3, TABLE_LIST = 4, TABLE_DESCRIPTION = 5, - GCP_ENTRY = 6 + GCP_ENTRY = 6, + FRAGMENT_INFO = 7 }; struct FileHeader { @@ -125,6 +126,20 @@ Uint32 SectionLength; Uint32 StartGCP; Uint32 StopGCP; + }; + + /** + * Fragment Info + */ + struct FragmentInfo { + Uint32 SectionType; + Uint32 SectionLength; + Uint32 TableId; + Uint32 FragmentNo; + Uint32 NoOfRecordsLow; + Uint32 NoOfRecordsHigh; + Uint32 FilePosLow; + Uint32 FilePosHigh; }; }; --- 1.15/ndb/src/kernel/blocks/backup/BackupInit.cpp 2006-04-26 15:59:42 +02:00 +++ 1.16/ndb/src/kernel/blocks/backup/BackupInit.cpp 2006-06-27 10:02:28 +02:00 @@ -97,6 +97,9 @@ addRecSignal(GSN_BACKUP_FRAGMENT_REQ, &Backup::execBACKUP_FRAGMENT_REQ); addRecSignal(GSN_BACKUP_FRAGMENT_REF, &Backup::execBACKUP_FRAGMENT_REF); addRecSignal(GSN_BACKUP_FRAGMENT_CONF, &Backup::execBACKUP_FRAGMENT_CONF); + + addRecSignal(GSN_BACKUP_FRAGMENT_COMPLETE_REP, + &Backup::execBACKUP_FRAGMENT_COMPLETE_REP); addRecSignal(GSN_STOP_BACKUP_REQ, &Backup::execSTOP_BACKUP_REQ); addRecSignal(GSN_STOP_BACKUP_REF, &Backup::execSTOP_BACKUP_REF); --- 1.26/ndb/tools/restore/Restore.cpp 2005-09-20 10:25:25 +02:00 +++ 1.27/ndb/tools/restore/Restore.cpp 2006-06-27 10:02:32 +02:00 @@ -80,7 +80,12 @@ RestoreMetaData::~RestoreMetaData(){ for(Uint32 i= 0; i < allTables.size(); i++) - delete allTables[i]; + { + TableS *table = allTables[i]; + for(Uint32 j= 0; j < table->m_fragmentInfo.size(); j++) + delete table->m_fragmentInfo[j]; + delete table; + } allTables.clear(); } @@ -111,6 +116,9 @@ } if(!readGCPEntry()) return 0; + + if(!readFragmentInfo()) + return 0; return 1; } @@ -192,6 +200,52 @@ return true; } +bool +RestoreMetaData::readFragmentInfo() +{ + BackupFormat::CtlFile::FragmentInfo fragInfo; + TableS * table = 0; + Uint32 tableId = RNIL; + + while (buffer_read(&fragInfo, 4, 2) == 2) + { + fragInfo.SectionType = ntohl(fragInfo.SectionType); + fragInfo.SectionLength = ntohl(fragInfo.SectionLength); + + if (fragInfo.SectionType != BackupFormat::FRAGMENT_INFO) + { + err << "readFragmentInfo invalid section type: " << + fragInfo.SectionType << endl; + return false; + } + + if (buffer_read(&fragInfo.TableId, (fragInfo.SectionLength-2)*4, 1) != 1) + { + err << "readFragmentInfo invalid section length: " << + fragInfo.SectionLength << endl; + return false; + } + + fragInfo.TableId = ntohl(fragInfo.TableId); + if (fragInfo.TableId != tableId) + { + tableId = fragInfo.TableId; + table = getTable(tableId); + } + + FragmentInfo * tmp = new FragmentInfo; + tmp->fragmentNo = ntohl(fragInfo.FragmentNo); + tmp->noOfRecords = ntohl(fragInfo.NoOfRecordsLow) + + (((Uint64)ntohl(fragInfo.NoOfRecordsHigh)) << 32); + tmp->filePosLow = ntohl(fragInfo.FilePosLow); + tmp->filePosHigh = ntohl(fragInfo.FilePosHigh); + + table->m_fragmentInfo.push_back(tmp); + table->m_noOfRecords += tmp->noOfRecords; + } + return true; +} + TableS::TableS(Uint32 version, NdbTableImpl* tableImpl) : m_dictTable(tableImpl) { @@ -199,6 +253,7 @@ m_noOfNullable = m_nullBitmaskSize = 0; m_auto_val_id= ~(Uint32)0; m_max_auto_val= 0; + m_noOfRecords= 0; backupVersion = version; for (int i = 0; i < tableImpl->getNoOfColumns(); i++) @@ -937,4 +992,5 @@ template class Vector<TableS*>; template class Vector<AttributeS*>; template class Vector<AttributeDesc*>; +template class Vector<FragmentInfo*>; --- 1.19/ndb/tools/restore/Restore.hpp 2005-06-08 16:48:54 +02:00 +++ 1.20/ndb/tools/restore/Restore.hpp 2006-06-27 10:02:32 +02:00 @@ -114,6 +114,14 @@ AttributeData * getData(int i) const; }; // class TupleS +struct FragmentInfo +{ + Uint32 fragmentNo; + Uint64 noOfRecords; + Uint32 filePosLow; + Uint32 filePosHigh; +}; + class TableS { friend class TupleS; @@ -136,6 +144,9 @@ int pos; + Uint64 m_noOfRecords; + Vector<FragmentInfo *> m_fragmentInfo; + void createAttr(NdbDictionary::Column *column); public: @@ -146,6 +157,9 @@ Uint32 getTableId() const { return m_dictTable->getTableId(); } + Uint32 getNoOfRecords() const { + return m_noOfRecords; + } /* void setMysqlTableName(char * tableName) { strpcpy(mysqlTableName, tableName); @@ -274,6 +288,7 @@ bool readMetaTableDesc(); bool readGCPEntry(); + bool readFragmentInfo(); Uint32 readMetaTableList(); Uint32 m_startGCP; --- 1.61/ndb/src/kernel/blocks/dbdict/Dbdict.cpp 2006-02-10 10:10:49 +01:00 +++ 1.62/ndb/src/kernel/blocks/dbdict/Dbdict.cpp 2006-06-27 10:02:28 +02:00 @@ -281,6 +281,10 @@ w.add(DictTabInfo::TableKValue, tablePtr.p->kValue); w.add(DictTabInfo::FragmentTypeVal, tablePtr.p->fragmentType); w.add(DictTabInfo::TableTypeVal, tablePtr.p->tableType); + w.add(DictTabInfo::MaxRowsLow, tablePtr.p->maxRowsLow); + w.add(DictTabInfo::MaxRowsHigh, tablePtr.p->maxRowsHigh); + w.add(DictTabInfo::MinRowsLow, tablePtr.p->minRowsLow); + w.add(DictTabInfo::MinRowsHigh, tablePtr.p->minRowsHigh); if(!signal) { @@ -1525,6 +1529,10 @@ tablePtr.p->minLoadFactor = 70; tablePtr.p->noOfPrimkey = 1; tablePtr.p->tupKeyLength = 1; + tablePtr.p->maxRowsLow = 0; + tablePtr.p->maxRowsHigh = 0; + tablePtr.p->minRowsLow = 0; + tablePtr.p->minRowsHigh = 0; tablePtr.p->storedTable = true; tablePtr.p->tableType = DictTabInfo::UserTable; tablePtr.p->primaryTableId = RNIL; @@ -4464,6 +4472,13 @@ Uint32 lhPageBits = 0; ::calcLHbits(&lhPageBits, &lhDistrBits, fragId, fragCount); + Uint64 maxRows = tabPtr.p->maxRowsLow + + (((Uint64)tabPtr.p->maxRowsHigh) << 32); + Uint64 minRows = tabPtr.p->minRowsLow + + (((Uint64)tabPtr.p->minRowsHigh) << 32); + maxRows = (maxRows + fragCount - 1) / fragCount; + minRows = (minRows + fragCount - 1) / fragCount; + { LqhFragReq* req = (LqhFragReq*)signal->getDataPtrSend(); req->senderData = senderData; @@ -4479,7 +4494,10 @@ req->lh3PageBits = 0; //lhPageBits; req->noOfAttributes = tabPtr.p->noOfAttributes; req->noOfNullAttributes = tabPtr.p->noOfNullBits; - req->noOfPagesToPreAllocate = 0; + req->maxRowsLow = maxRows & 0xFFFFFFFF; + req->maxRowsHigh = maxRows >> 32; + req->minRowsLow = minRows & 0xFFFFFFFF; + req->minRowsHigh = minRows >> 32; req->schemaVersion = tabPtr.p->tableVersion; Uint32 keyLen = tabPtr.p->tupKeyLength; req->keyLength = keyLen; // wl-2066 no more "long keys" @@ -4487,8 +4505,7 @@ req->noOfKeyAttr = tabPtr.p->noOfPrimkey; req->noOfNewAttr = 0; - // noOfCharsets passed to TUP in upper half - req->noOfNewAttr |= (tabPtr.p->noOfCharsets << 16); + req->noOfCharsets = tabPtr.p->noOfCharsets; req->checksumIndicator = 1; req->noOfAttributeGroups = 1; req->GCPIndicator = 0; @@ -5054,6 +5071,15 @@ tablePtr.p->tableType = (DictTabInfo::TableType)tableDesc.TableType; tablePtr.p->kValue = tableDesc.TableKValue; tablePtr.p->fragmentCount = tableDesc.FragmentCount; + tablePtr.p->maxRowsLow = tableDesc.MaxRowsLow; + tablePtr.p->maxRowsHigh = tableDesc.MaxRowsHigh; + tablePtr.p->minRowsLow = tableDesc.MinRowsLow; + tablePtr.p->minRowsHigh = tableDesc.MinRowsHigh; + + Uint64 maxRows = + (((Uint64)tablePtr.p->maxRowsHigh) << 32) + tablePtr.p->maxRowsLow; + Uint64 minRows = + (((Uint64)tablePtr.p->minRowsHigh) << 32) + tablePtr.p->minRowsLow; tablePtr.p->frmLen = tableDesc.FrmLen; memcpy(tablePtr.p->frmData, tableDesc.FrmData, tableDesc.FrmLen); --- 1.16/ndb/src/kernel/blocks/dbdict/Dbdict.hpp 2005-09-15 15:00:32 +02:00 +++ 1.17/ndb/src/kernel/blocks/dbdict/Dbdict.hpp 2006-06-27 10:02:28 +02:00 @@ -131,6 +131,10 @@ * on disk. Index trigger ids are volatile. */ struct TableRecord : public MetaData::Table { + Uint32 maxRowsLow; + Uint32 maxRowsHigh; + Uint32 minRowsLow; + Uint32 minRowsHigh; /**************************************************** * Support variables for table handling ****************************************************/ --- 1.37/ndb/src/kernel/blocks/dblqh/Dblqh.hpp 2005-12-13 11:51:01 +01:00 +++ 1.38/ndb/src/kernel/blocks/dblqh/Dblqh.hpp 2006-06-27 10:02:28 +02:00 @@ -443,7 +443,6 @@ UintR dictConnectptr; UintR fragmentPtr; UintR nextAddfragrec; - UintR noOfAllocPages; UintR schemaVer; UintR tup1Connectptr; UintR tup2Connectptr; @@ -465,12 +464,17 @@ Uint16 totalAttrReceived; Uint16 fragCopyCreation; Uint16 noOfKeyAttr; - Uint32 noOfNewAttr; // noOfCharsets in upper half + Uint16 noOfNewAttr; + Uint16 noOfCharsets; Uint16 noOfAttributeGroups; Uint16 lh3DistrBits; Uint16 tableType; Uint16 primaryTableId; - };// Size 108 bytes + Uint32 maxRowsLow; + Uint32 maxRowsHigh; + Uint32 minRowsLow; + Uint32 minRowsHigh; + };// Size 124 bytes typedef Ptr<AddFragRecord> AddFragRecordPtr; /* $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ */ --- 1.87/ndb/src/kernel/blocks/dblqh/DblqhMain.cpp 2006-06-20 19:13:39 +02:00 +++ 1.88/ndb/src/kernel/blocks/dblqh/DblqhMain.cpp 2006-06-27 10:02:29 +02:00 @@ -939,12 +939,16 @@ Uint8 tlh = req->lh3PageBits; Uint32 tnoOfAttr = req->noOfAttributes; Uint32 tnoOfNull = req->noOfNullAttributes; - Uint32 noOfAlloc = req->noOfPagesToPreAllocate; + Uint32 maxRowsLow = req->maxRowsLow; + Uint32 maxRowsHigh = req->maxRowsHigh; + Uint32 minRowsLow = req->minRowsLow; + Uint32 minRowsHigh = req->minRowsHigh; Uint32 tschemaVersion = req->schemaVersion; Uint32 ttupKeyLength = req->keyLength; Uint32 nextLcp = req->nextLCP; Uint32 noOfKeyAttr = req->noOfKeyAttr; Uint32 noOfNewAttr = req->noOfNewAttr; + Uint32 noOfCharsets = req->noOfCharsets; Uint32 checksumIndicator = req->checksumIndicator; Uint32 noOfAttributeGroups = req->noOfAttributeGroups; Uint32 gcpIndicator = req->GCPIndicator; @@ -1042,7 +1046,10 @@ addfragptr.p->m_senderAttrPtr = RNIL; addfragptr.p->noOfAttr = tnoOfAttr; addfragptr.p->noOfNull = tnoOfNull; - addfragptr.p->noOfAllocPages = noOfAlloc; + addfragptr.p->maxRowsLow = maxRowsLow; + addfragptr.p->maxRowsHigh = maxRowsHigh; + addfragptr.p->minRowsLow = minRowsLow; + addfragptr.p->minRowsHigh = minRowsHigh; addfragptr.p->tabId = tabptr.i; addfragptr.p->totalAttrReceived = 0; addfragptr.p->attrSentToTup = ZNIL;/* TO FIND PROGRAMMING ERRORS QUICKLY */ @@ -1052,6 +1059,7 @@ addfragptr.p->addfragErrorCode = 0; addfragptr.p->noOfKeyAttr = noOfKeyAttr; addfragptr.p->noOfNewAttr = noOfNewAttr; + addfragptr.p->noOfCharsets = noOfCharsets; addfragptr.p->checksumIndicator = checksumIndicator; addfragptr.p->noOfAttributeGroups = noOfAttributeGroups; addfragptr.p->GCPIndicator = gcpIndicator; @@ -1221,47 +1229,56 @@ ptrCheckGuard(fragptr, cfragrecFileSize, fragrecord); if (addfragptr.p->addfragStatus == AddFragRecord::WAIT_TWO_TUP || addfragptr.p->addfragStatus == AddFragRecord::WAIT_ONE_TUP) { + TupFragReq* const tupFragReq = (TupFragReq*)signal->getDataPtrSend(); if (DictTabInfo::isTable(addfragptr.p->tableType) || DictTabInfo::isHashIndex(addfragptr.p->tableType)) { jam(); - signal->theData[0] = addfragptr.i; - signal->theData[1] = cownref; - signal->theData[2] = 0; /* ADD TABLE */ - signal->theData[3] = addfragptr.p->tabId; - signal->theData[4] = addfragptr.p->noOfAttr; - signal->theData[5] = + tupFragReq->userPtr = addfragptr.i; + tupFragReq->userRef = cownref; + tupFragReq->reqInfo = 0; /* ADD TABLE */ + tupFragReq->tableId = addfragptr.p->tabId; + tupFragReq->noOfAttr = addfragptr.p->noOfAttr; + tupFragReq->fragId = addfragptr.p->addfragStatus == AddFragRecord::WAIT_TWO_TUP ? addfragptr.p->fragid1 : addfragptr.p->fragid2; - signal->theData[6] = (addfragptr.p->noOfAllocPages >> 1) + 1; - signal->theData[7] = addfragptr.p->noOfNull; - signal->theData[8] = addfragptr.p->schemaVer; - signal->theData[9] = addfragptr.p->noOfKeyAttr; - signal->theData[10] = addfragptr.p->noOfNewAttr; - signal->theData[11] = addfragptr.p->checksumIndicator; - signal->theData[12] = addfragptr.p->noOfAttributeGroups; - signal->theData[13] = addfragptr.p->GCPIndicator; + tupFragReq->maxRowsLow = addfragptr.p->maxRowsLow; + tupFragReq->maxRowsHigh = addfragptr.p->maxRowsHigh; + tupFragReq->minRowsLow = addfragptr.p->minRowsLow; + tupFragReq->minRowsHigh = addfragptr.p->minRowsHigh; + tupFragReq->noOfNullAttr = addfragptr.p->noOfNull; + tupFragReq->schemaVersion = addfragptr.p->schemaVer; + tupFragReq->noOfKeyAttr = addfragptr.p->noOfKeyAttr; + tupFragReq->noOfNewAttr = addfragptr.p->noOfNewAttr; + tupFragReq->noOfCharsets = addfragptr.p->noOfCharsets; + tupFragReq->checksumIndicator = addfragptr.p->checksumIndicator; + tupFragReq->noOfAttributeGroups = addfragptr.p->noOfAttributeGroups; + tupFragReq->globalCheckpointIdIndicator = addfragptr.p->GCPIndicator; sendSignal(fragptr.p->tupBlockref, GSN_TUPFRAGREQ, signal, TupFragReq::SignalLength, JBB); return; } if (DictTabInfo::isOrderedIndex(addfragptr.p->tableType)) { jam(); - signal->theData[0] = addfragptr.i; - signal->theData[1] = cownref; - signal->theData[2] = 0; /* ADD TABLE */ - signal->theData[3] = addfragptr.p->tabId; - signal->theData[4] = 1; /* ordered index: one array attr */ - signal->theData[5] = + tupFragReq->userPtr = addfragptr.i; + tupFragReq->userRef = cownref; + tupFragReq->reqInfo = 0; /* ADD TABLE */ + tupFragReq->tableId = addfragptr.p->tabId; + tupFragReq->noOfAttr = 1; /* ordered index: one array attr */ + tupFragReq->fragId = addfragptr.p->addfragStatus == AddFragRecord::WAIT_TWO_TUP ? addfragptr.p->fragid1 : addfragptr.p->fragid2; - signal->theData[6] = (addfragptr.p->noOfAllocPages >> 1) + 1; - signal->theData[7] = 0; /* ordered index: no nullable */ - signal->theData[8] = addfragptr.p->schemaVer; - signal->theData[9] = 1; /* ordered index: one key */ - signal->theData[10] = addfragptr.p->noOfNewAttr; - signal->theData[11] = addfragptr.p->checksumIndicator; - signal->theData[12] = addfragptr.p->noOfAttributeGroups; - signal->theData[13] = addfragptr.p->GCPIndicator; + tupFragReq->maxRowsLow = addfragptr.p->maxRowsLow; + tupFragReq->maxRowsHigh = addfragptr.p->maxRowsHigh; + tupFragReq->minRowsLow = addfragptr.p->minRowsLow; + tupFragReq->minRowsHigh = addfragptr.p->minRowsHigh; + tupFragReq->noOfNullAttr = 0; /* ordered index: no nullable */ + tupFragReq->schemaVersion = addfragptr.p->schemaVer; + tupFragReq->noOfKeyAttr = 1; /* ordered index: one key */ + tupFragReq->noOfNewAttr = addfragptr.p->noOfNewAttr; + tupFragReq->noOfCharsets = addfragptr.p->noOfCharsets; + tupFragReq->checksumIndicator = addfragptr.p->checksumIndicator; + tupFragReq->noOfAttributeGroups = addfragptr.p->noOfAttributeGroups; + tupFragReq->globalCheckpointIdIndicator = addfragptr.p->GCPIndicator; sendSignal(fragptr.p->tupBlockref, GSN_TUPFRAGREQ, signal, TupFragReq::SignalLength, JBB); return; @@ -1580,28 +1597,35 @@ { fragptr.i = addfragptr.p->fragmentPtr; ptrCheckGuard(fragptr, cfragrecFileSize, fragrecord); - signal->theData[0] = (Uint32)-1; if (addfragptr.p->tup1Connectptr != RNIL) { jam(); - signal->theData[1] = addfragptr.p->tup1Connectptr; + TupFragReq* const tupFragReq = (TupFragReq*)signal->getDataPtrSend(); + tupFragReq->userPtr = (Uint32)-1; + tupFragReq->userRef = addfragptr.p->tup1Connectptr; sendSignal(fragptr.p->tupBlockref, GSN_TUPFRAGREQ, signal, 2, JBB); addfragptr.p->tup1Connectptr = RNIL; } if (addfragptr.p->tup2Connectptr != RNIL) { jam(); - signal->theData[1] = addfragptr.p->tup2Connectptr; + TupFragReq* const tupFragReq = (TupFragReq*)signal->getDataPtrSend(); + tupFragReq->userPtr = (Uint32)-1; + tupFragReq->userRef = addfragptr.p->tup2Connectptr; sendSignal(fragptr.p->tupBlockref, GSN_TUPFRAGREQ, signal, 2, JBB); addfragptr.p->tup2Connectptr = RNIL; } if (addfragptr.p->tux1Connectptr != RNIL) { jam(); - signal->theData[1] = addfragptr.p->tux1Connectptr; + TuxFragReq* const tuxFragReq = (TuxFragReq*)signal->getDataPtrSend(); + tuxFragReq->userPtr = (Uint32)-1; + tuxFragReq->userRef = addfragptr.p->tux1Connectptr; sendSignal(fragptr.p->tuxBlockref, GSN_TUXFRAGREQ, signal, 2, JBB); addfragptr.p->tux1Connectptr = RNIL; } if (addfragptr.p->tux2Connectptr != RNIL) { jam(); - signal->theData[1] = addfragptr.p->tux2Connectptr; + TuxFragReq* const tuxFragReq = (TuxFragReq*)signal->getDataPtrSend(); + tuxFragReq->userPtr = (Uint32)-1; + tuxFragReq->userRef = addfragptr.p->tux2Connectptr; sendSignal(fragptr.p->tuxBlockref, GSN_TUXFRAGREQ, signal, 2, JBB); addfragptr.p->tux2Connectptr = RNIL; } --- 1.25/ndb/src/kernel/blocks/dbtup/Dbtup.hpp 2005-09-21 17:37:31 +02:00 +++ 1.26/ndb/src/kernel/blocks/dbtup/Dbtup.hpp 2006-06-27 10:02:30 +02:00 @@ -496,7 +496,8 @@ typedef Ptr<DiskBufferSegmentInfo> DiskBufferSegmentInfoPtr; struct Fragoperrec { - bool definingFragment; + Uint64 minRows; + Uint64 maxRows; Uint32 nextFragoprec; Uint32 lqhPtrFrag; Uint32 fragidFrag; @@ -509,6 +510,7 @@ Uint32 charsetIndex; BlockReference lqhBlockrefFrag; bool inUse; + bool definingFragment; }; typedef Ptr<Fragoperrec> FragoperrecPtr; @@ -560,6 +562,7 @@ Uint32 currentPageRange; Uint32 rootPageRange; Uint32 noOfPages; + Uint32 noOfPagesToGrow; Uint32 emptyPrimPage; Uint32 firstusedOprec; --- 1.13/ndb/src/kernel/blocks/dbtup/DbtupMeta.cpp 2005-09-01 00:19:31 +02:00 +++ 1.14/ndb/src/kernel/blocks/dbtup/DbtupMeta.cpp 2006-06-27 10:02:30 +02:00 @@ -41,7 +41,8 @@ { ljamEntry(); - if (signal->theData[0] == (Uint32)-1) { + TupFragReq* tupFragReq = (TupFragReq*)signal->getDataPtr(); + if (tupFragReq->userPtr == (Uint32)-1) { ljam(); abortAddFragOp(signal); return; @@ -51,30 +52,34 @@ FragrecordPtr regFragPtr; TablerecPtr regTabPtr; - Uint32 userptr = signal->theData[0]; - Uint32 userblockref = signal->theData[1]; - Uint32 reqinfo = signal->theData[2]; - regTabPtr.i = signal->theData[3]; - Uint32 noOfAttributes = signal->theData[4]; - Uint32 fragId = signal->theData[5]; - Uint32 noOfNullAttr = signal->theData[7]; - /* Uint32 schemaVersion = signal->theData[8];*/ - Uint32 noOfKeyAttr = signal->theData[9]; - - Uint32 noOfNewAttr = (signal->theData[10] & 0xFFFF); - /* DICT sends number of character sets in upper half */ - Uint32 noOfCharsets = (signal->theData[10] >> 16); - - Uint32 checksumIndicator = signal->theData[11]; - Uint32 noOfAttributeGroups = signal->theData[12]; - Uint32 globalCheckpointIdIndicator = signal->theData[13]; + Uint32 userptr = tupFragReq->userPtr; + Uint32 userblockref = tupFragReq->userRef; + Uint32 reqinfo = tupFragReq->reqInfo; + regTabPtr.i = tupFragReq->tableId; + Uint32 noOfAttributes = tupFragReq->noOfAttr; + Uint32 fragId = tupFragReq->fragId; + Uint32 noOfNullAttr = tupFragReq->noOfNullAttr; + /* Uint32 schemaVersion = tupFragReq->schemaVersion;*/ + Uint32 noOfKeyAttr = tupFragReq->noOfKeyAttr; + + Uint32 noOfNewAttr = tupFragReq->noOfNewAttr; + Uint32 noOfCharsets = tupFragReq->noOfCharsets; + + Uint32 checksumIndicator = tupFragReq->checksumIndicator; + Uint32 noOfAttributeGroups = tupFragReq->noOfAttributeGroups; + Uint32 globalCheckpointIdIndicator = tupFragReq->globalCheckpointIdIndicator; + + Uint64 maxRows = + (((Uint64)tupFragReq->maxRowsHigh) << 32) + tupFragReq->maxRowsLow; + Uint64 minRows = + (((Uint64)tupFragReq->minRowsHigh) << 32) + tupFragReq->minRowsLow; #ifndef VM_TRACE // config mismatch - do not crash if release compiled if (regTabPtr.i >= cnoOfTablerec) { ljam(); - signal->theData[0] = userptr; - signal->theData[1] = 800; + tupFragReq->userPtr = userptr; + tupFragReq->userRef = 800; sendSignal(userblockref, GSN_TUPFRAGREF, signal, 2, JBB); return; } @@ -83,8 +88,8 @@ ptrCheckGuard(regTabPtr, cnoOfTablerec, tablerec); if (cfirstfreeFragopr == RNIL) { ljam(); - signal->theData[0] = userptr; - signal->theData[1] = ZNOFREE_FRAGOP_ERROR; + tupFragReq->userPtr = userptr; + tupFragReq->userRef = ZNOFREE_FRAGOP_ERROR; sendSignal(userblockref, GSN_TUPFRAGREF, signal, 2, JBB); return; }//if @@ -100,6 +105,9 @@ fragOperPtr.p->noOfNewAttrCount = noOfNewAttr; fragOperPtr.p->charsetIndex = 0; fragOperPtr.p->currNullBit = 0; + // remove in 5.1, 2 fragments per fragment in 5.0 + fragOperPtr.p->minRows = (minRows + 1)/2; + fragOperPtr.p->maxRows = (maxRows + 1)/2; ndbrequire(reqinfo == ZADDFRAG); @@ -141,16 +149,6 @@ regFragPtr.p->fragmentId = fragId; regFragPtr.p->checkpointVersion = RNIL; - Uint32 noAllocatedPages = 2; - noAllocatedPages = allocFragPages(regFragPtr.p, noAllocatedPages); - - if (noAllocatedPages == 0) { - ljam(); - terrorCode = ZNO_PAGES_ALLOCATED_ERROR; - fragrefuse3Lab(signal, fragOperPtr, regFragPtr, regTabPtr.p, fragId); - return; - }//if - if (ERROR_INSERTED(4007) && regTabPtr.p->fragid[0] == fragId || ERROR_INSERTED(4008) && regTabPtr.p->fragid[1] == fragId) { ljam(); @@ -407,6 +405,27 @@ CLEAR_ERROR_INSERT_VALUE; return; } + + if (lastAttr) + { + ljam(); + Uint32 noRowsPerPage = ZWORDS_ON_PAGE/regTabPtr.p->tupheadsize; + Uint32 noAllocatedPages = + (fragOperPtr.p->minRows + noRowsPerPage - 1 )/ noRowsPerPage; + if (fragOperPtr.p->minRows == 0) + noAllocatedPages = 2; + else if (noAllocatedPages == 0) + noAllocatedPages = 2; + noAllocatedPages = allocFragPages(regFragPtr.p, noAllocatedPages); + + if (noAllocatedPages == 0) { + ljam(); + terrorCode = ZNO_PAGES_ALLOCATED_ERROR; + addattrrefuseLab(signal, regFragPtr, fragOperPtr, regTabPtr.p, fragId); + return; + }//if + } + /* **************************************************************** */ /* ************** TUP_ADD_ATTCONF ****************** */ /* **************************************************************** */ --- 1.1/ndb/src/kernel/blocks/dbtup/DbtupPageMap.cpp 2004-04-14 10:24:20 +02:00 +++ 1.2/ndb/src/kernel/blocks/dbtup/DbtupPageMap.cpp 2006-06-27 10:02:30 +02:00 @@ -332,6 +332,7 @@ regFragPtr->rootPageRange = RNIL; regFragPtr->currentPageRange = RNIL; regFragPtr->noOfPages = 0; + regFragPtr->noOfPagesToGrow = 2; regFragPtr->nextStartRange = 0; }//initFragRange() @@ -393,9 +394,10 @@ void Dbtup::allocMoreFragPages(Fragrecord* const regFragPtr) { - Uint32 noAllocPages = regFragPtr->noOfPages >> 3; // 12.5% - noAllocPages += regFragPtr->noOfPages >> 4; // 6.25% + Uint32 noAllocPages = regFragPtr->noOfPagesToGrow >> 3; // 12.5% + noAllocPages += regFragPtr->noOfPagesToGrow >> 4; // 6.25% noAllocPages += 2; + regFragPtr->noOfPagesToGrow += noAllocPages; /* -----------------------------------------------------------------*/ // We will grow by 18.75% plus two more additional pages to grow // a little bit quicker in the beginning. --- 1.101/ndb/src/mgmsrv/MgmtSrvr.cpp 2006-06-20 10:27:46 +02:00 +++ 1.102/ndb/src/mgmsrv/MgmtSrvr.cpp 2006-06-27 10:02:30 +02:00 @@ -2380,13 +2380,19 @@ event.Event = BackupEvent::BackupCompleted; event.Completed.BackupId = rep->backupId; - event.Completed.NoOfBytes = rep->noOfBytes; + event.Completed.NoOfBytes = rep->noOfBytesLow; event.Completed.NoOfLogBytes = rep->noOfLogBytes; - event.Completed.NoOfRecords = rep->noOfRecords; + event.Completed.NoOfRecords = rep->noOfRecordsLow; event.Completed.NoOfLogRecords = rep->noOfLogRecords; event.Completed.stopGCP = rep->stopGCP; event.Completed.startGCP = rep->startGCP; event.Nodes = rep->nodes; + + if (signal->header.theLength >= BackupCompleteRep::SignalLength) + { + event.Completed.NoOfBytes += ((Uint64)rep->noOfBytesHigh) << 32; + event.Completed.NoOfRecords += ((Uint64)rep->noOfRecordsHigh) << 32; + } backupId = rep->backupId; return 0; --- 1.45/ndb/src/mgmsrv/MgmtSrvr.hpp 2006-06-16 13:00:57 +02:00 +++ 1.46/ndb/src/mgmsrv/MgmtSrvr.hpp 2006-06-27 10:02:31 +02:00 @@ -323,9 +323,9 @@ Uint32 ErrorCode; } FailedToStart ; struct { + Uint64 NoOfBytes; + Uint64 NoOfRecords; Uint32 BackupId; - Uint32 NoOfBytes; - Uint32 NoOfRecords; Uint32 NoOfLogBytes; Uint32 NoOfLogRecords; Uint32 startGCP; --- 1.37/ndb/src/ndbapi/NdbDictionary.cpp 2006-05-24 10:57:27 +02:00 +++ 1.38/ndb/src/ndbapi/NdbDictionary.cpp 2006-06-27 10:02:32 +02:00 @@ -385,6 +385,30 @@ return m_impl.m_noOfKeys; } +void +NdbDictionary::Table::setMaxRows(Uint64 maxRows) +{ + m_impl.m_max_rows = maxRows; +} + +Uint64 +NdbDictionary::Table::getMaxRows() const +{ + return m_impl.m_max_rows; +} + +void +NdbDictionary::Table::setMinRows(Uint64 minRows) +{ + m_impl.m_min_rows = minRows; +} + +Uint64 +NdbDictionary::Table::getMinRows() const +{ + return m_impl.m_min_rows; +} + const char* NdbDictionary::Table::getPrimaryKey(int no) const { int count = 0; --- 1.93/ndb/src/ndbapi/NdbDictionaryImpl.cpp 2006-06-12 14:26:02 +02:00 +++ 1.94/ndb/src/ndbapi/NdbDictionaryImpl.cpp 2006-06-27 10:02:32 +02:00 @@ -319,6 +319,8 @@ m_noOfDistributionKeys= 0; m_noOfBlobs= 0; m_replicaCount= 0; + m_min_rows = 0; + m_max_rows = 0; } bool @@ -416,6 +418,9 @@ m_version = org.m_version; m_status = org.m_status; + + m_max_rows = org.m_max_rows; + m_min_rows = org.m_min_rows; } void NdbTableImpl::setName(const char * name) @@ -1302,6 +1307,12 @@ fragmentTypeMapping, (Uint32)NdbDictionary::Object::FragUndefined); + Uint64 max_rows = ((Uint64)tableDesc.MaxRowsHigh) << 32; + max_rows += tableDesc.MaxRowsLow; + impl->m_max_rows = max_rows; + Uint64 min_rows = ((Uint64)tableDesc.MinRowsHigh) << 32; + min_rows += tableDesc.MinRowsLow; + impl->m_min_rows = min_rows; impl->m_logging = tableDesc.TableLoggedFlag; impl->m_kvalue = tableDesc.TableKValue; impl->m_minLoadFactor = tableDesc.MinLoadFactor; @@ -1630,7 +1641,16 @@ tmpTab.MaxLoadFactor = impl.m_maxLoadFactor; tmpTab.TableType = DictTabInfo::UserTable; tmpTab.NoOfAttributes = sz; - + tmpTab.MaxRowsHigh = (Uint32)(impl.m_max_rows >> 32); + tmpTab.MaxRowsLow = (Uint32)(impl.m_max_rows & 0xFFFFFFFF); + tmpTab.MinRowsHigh = (Uint32)(impl.m_min_rows >> 32); + tmpTab.MinRowsLow = (Uint32)(impl.m_min_rows & 0xFFFFFFFF); + + Uint64 maxRows = + (((Uint64)tmpTab.MaxRowsHigh) << 32) + tmpTab.MaxRowsLow; + Uint64 minRows = + (((Uint64)tmpTab.MinRowsHigh) << 32) + tmpTab.MinRowsLow; + tmpTab.FragmentType = getKernelConstant(impl.m_fragmentType, fragmentTypeMapping, DictTabInfo::AllNodesSmallTable); --- 1.38/ndb/src/ndbapi/NdbDictionaryImpl.hpp 2006-06-12 14:26:02 +02:00 +++ 1.39/ndb/src/ndbapi/NdbDictionaryImpl.hpp 2006-06-27 10:02:32 +02:00 @@ -130,6 +130,9 @@ Uint32 m_hashpointerValue; Vector<Uint16> m_fragments; + Uint64 m_max_rows; + Uint64 m_min_rows; + bool m_logging; int m_kvalue; int m_minLoadFactor; --- 1.261/sql/ha_ndbcluster.cc 2006-06-20 00:07:51 +02:00 +++ 1.262/sql/ha_ndbcluster.cc 2006-06-27 10:02:33 +02:00 @@ -4111,7 +4111,11 @@ static void ndb_set_fragmentation(NDBTAB &tab, TABLE *form, uint pk_length) { - if (form->s->max_rows == (ha_rows) 0) /* default setting, don't set fragmentation */ + ha_rows max_rows= form->s->max_rows; + ha_rows min_rows= form->s->min_rows; + if (max_rows < min_rows) + max_rows= min_rows; + if (max_rows == (ha_rows)0) /* default setting, don't set fragmentation */ return; /** * get the number of fragments right @@ -4129,7 +4133,6 @@ acc_row_size+= 4 + /*safety margin*/ 4; #endif ulonglong acc_fragment_size= 512*1024*1024; - ulonglong max_rows= form->s->max_rows; #if MYSQL_VERSION_ID >= 50100 no_fragments= (max_rows*acc_row_size)/acc_fragment_size+1; #else @@ -4153,6 +4156,9 @@ ftype= NDBTAB::FragAllSmall; tab.setFragmentType(ftype); } + tab.setMaxRows(max_rows); + tab.setMinRows(min_rows); + fprintf(stderr, "max/min %llu %llu\n", max_rows, min_rows); } int ha_ndbcluster::create(const char *name, --- 1.23/ndb/tools/restore/consumer_restore.cpp 2006-05-17 12:30:57 +02:00 +++ 1.24/ndb/tools/restore/consumer_restore.cpp 2006-06-27 10:02:32 +02:00 @@ -193,6 +193,16 @@ copy.setName(split[2].c_str()); + /* + update min and max rows to reflect the table, this to + ensure that memory is allocated properly in the ndb kernel + */ + copy.setMinRows(table.getNoOfRecords()); + if (table.getNoOfRecords() > copy.getMaxRows()) + { + copy.setMaxRows(table.getNoOfRecords()); + } + if (dict->createTable(copy) == -1) { err << "Create table " << table.getTableName() << " failed: "
| Thread | ||
|---|---|---|
| • bk commit into 5.0 tree (tomas:1.2199) BUG#19852 | tomas | 27 Jun |
© 1995-2008 MySQL AB. All rights reserved.
Page generated in 0.829 sec. using MySQL 5.1.14-beta-log