#At file:///home/cpowers/work/dev/dev-06/mysql/
3046 Christopher Powers 2009-03-03
Bug #42651 "Regression: falcon_bug_22169-big started to fail with error 305"
Bug #33177 "Table creation fails after error 305 and tablespace change"
Bug #32838 "Falcon; error 1296 : Got error 305 'record memory is exhausted'"
The fix for these bugs is the first of several improvements
to Falcon's memory management (Worklog TBD).
Falcon out-of-memory errors are caused by a combination of things.
Recent improvements to the Scavenger and to the Backlogging subsystem
(Bug#42592) have contributed to the resolution of these bugs, however,
certain operations can still fill the record cache to the point where
scavenging is ineffective.
Scavenging efficiency will be greatly improved by allocating record
data and metadata separately. The record cache now stores only
actual record data, and Record and RecordVersion objects (metadata)
are allocated from separate memory pools.
The metadata memory pools are completely homogeneous, with no memory
fragmentation. The record cache will also be far less fragmented,
because large blocks of record data will no longer be interspersed
with very small blocks of object data.
Decoupling the data and metadata will also greatly reduce the number of
out-of-memory conditions--typically seen during large inserts and
updates--because the memory pools are allowed to grow independently.
These memory pools may flucuate considerably during massive transactions,
depending upon the record makeup and type of operation. This flucuation,
however, serves only to emphasize the value managing these memory pools
separately.
One side-effect of this change is that, while the record cache max size
remains fixed, the record metadata caches can grow unbounded. Although
this is not unprecedented (Falcon's general purpose memory pool has
always been unbounded), one remaining challenge is to ensure that
the Falcon memory manager releases resources back to the system as
soon as possible.
modified:
storage/falcon/Database.cpp
storage/falcon/Database.h
storage/falcon/MemControl.cpp
storage/falcon/MemControl.h
storage/falcon/MemMgr.cpp
storage/falcon/MemMgr.h
storage/falcon/MemoryManager.h
storage/falcon/RecordScavenge.cpp
storage/falcon/RecordVersion.cpp
storage/falcon/Scavenger.cpp
storage/falcon/Table.cpp
per-file messages:
storage/falcon/Database.cpp
Access the memory managers via the MemControl class
Database::Database()
- Set pointers to the memory managers for record data and
record metadata (Record and RecordVersion objects)
Database::scavengeRecords()
- Check scavenge threshold against active memory in record data pool
Database::setLowMemory()
- Added spaceNeeded parameter
storage/falcon/Database.h
Increased SCAVENGE_WAIT_MS from 20 to 50 ms
Added MemMgr pointers: recordDataPool, recordPool and recordVersionPool
Added Database::lowMemoryCount - max scavenge cycles for low memory to stay
true below threshold
storage/falcon/MemControl.cpp
MemControl
- ::maxMemory overrides combined memory limits of pools controlled
by MemControl object
- Added getCurrentMemory(). Can get current memory of combined
pools within MemControl, or individual pools
storage/falcon/MemControl.h
Added getCurrentMemory()
storage/falcon/MemMgr.cpp
Defined memory managers for record data, record objects and record
version objects.
Defined unique memory managers ids for the separate memory pools.
storage/falcon/MemMgr.h
MemMgr
- Added id
- Added mgrId parameter to constructor
- Added maxMemory to allow a memory limit for individual pools
storage/falcon/MemoryManager.h
Added MemMgr identifiers
Defined two MemControl objects: one for general purpose
memory control, the other for record data and metadata
storage/falcon/RecordScavenge.cpp
Simplified RecordScavenge::computeThreshold()
Set startingActiveMemory to that of the record data pool
storage/falcon/RecordVersion.cpp
Comments
storage/falcon/Scavenger.cpp
Scavenger::execute()
- Set 'forced' flag during low memory
storage/falcon/Table.cpp
Allocate record objects from the record object pool
Allocate recordVersion objects from the record version object pool
=== modified file 'storage/falcon/Database.cpp'
--- a/storage/falcon/Database.cpp 2009-02-26 20:04:31 +0000
+++ b/storage/falcon/Database.cpp 2009-03-03 07:09:29 +0000
@@ -1,4 +1,4 @@
-/* Copyright � 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
+/* Copyright (C) 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -72,6 +72,7 @@
#include "InfoTable.h"
#include "MemoryManager.h"
#include "MemMgr.h"
+#include "MemControl.h"
#include "RecordScavenge.h"
#include "RecordSection.h"
#include "LogStream.h"
@@ -117,6 +118,7 @@ static const char THIS_FILE[]=__FILE__;
#define STATEMENT_RETIREMENT_AGE 60
#define RECORD_RETIREMENT_AGE 60
+#define MAX_LOW_MEMORY 10
extern uint falcon_debug_trace;
@@ -484,8 +486,10 @@ Database::Database(const char *dbName, C
scavengeCycle = 0;
serialLogBlockSize = configuration->serialLogBlockSize;
longSync = false;
- recordDataPool = MemMgrGetFixedPool(MemMgrPoolRecordData);
- //recordObjectPool = MemMgrGetFixedPool(MemMgrPoolRecordObject);
+ recordMemoryControl = MemMgrGetControl(MemMgrControlRecord);
+ recordPool = MemMgrGetFixedPool(MemMgrRecord);
+ recordVersionPool = MemMgrGetFixedPool(MemMgrRecordVersion);
+ recordDataPool = MemMgrGetFixedPool(MemMgrRecordData);
syncObject.setName("Database::syncObject");
syncTables.setName("Database::syncTables");
syncStatements.setName("Database::syncStatements");
@@ -497,7 +501,6 @@ Database::Database(const char *dbName, C
IO::deleteFile(BACKLOG_FILE);
}
-
void Database::start()
{
symbolManager = new SymbolManager;
@@ -1826,7 +1829,7 @@ void Database::scavengeRecords(bool forc
// Take inventory of the record cache and prune invisible record versions
pruneRecords(&recordScavenge);
- recordScavenge.prunedActiveMemory = recordDataPool->activeMemory;
+ recordScavenge.prunedActiveMemory = recordMemoryControl->getCurrentMemory(MemMgrRecordData);
recordScavenge.pruneStop = deltaTime;
syncScavenger.unlock(); // take a breath!
@@ -1834,15 +1837,20 @@ void Database::scavengeRecords(bool forc
syncScavenger.lock(Exclusive);
retireRecords(&recordScavenge);
- recordScavenge.retiredActiveMemory = recordDataPool->activeMemory;
+ recordScavenge.retiredActiveMemory = recordMemoryControl->getCurrentMemory(MemMgrRecordData);
recordScavenge.retireStop = deltaTime;
// Enable backlogging if memory is low
if (recordScavenge.retiredActiveMemory > recordScavengeFloor)
- setLowMemory();
+ if (!lowMemory)
+ setLowMemory(recordScavenge.retiredActiveMemory - recordScavengeFloor);
else
- clearLowMemory();
+ {
+ if (lowMemoryCount)
+ if (--lowMemoryCount == 0)
+ clearLowMemory();
+ }
recordScavenge.print();
// Log::log(analyze(analyzeRecordLeafs));
@@ -1859,7 +1867,7 @@ void Database::scavengeRecords(bool forc
Sync syncMem(&syncMemory, "Database::checkRecordScavenge");
syncMem.lock(Exclusive);
- lastActiveMemoryChecked = lastGenerationMemory = recordDataPool->activeMemory;
+ lastActiveMemoryChecked = lastGenerationMemory = recordMemoryControl->getCurrentMemory(MemMgrRecordData);
}
// Take inventory of the record cache and prune invisible record versions
@@ -1892,7 +1900,7 @@ void Database::retireRecords(RecordScave
// Scavenge if we passed the upper limit or if a forced scavenge
// was requested.
- if (recordDataPool->activeMemory < recordScavengeThreshold
+ if (recordMemoryControl->getCurrentMemory(MemMgrRecordData) < recordScavengeThreshold
&& !recordScavenge->forced)
return;
@@ -1902,7 +1910,7 @@ void Database::retireRecords(RecordScave
Sync syncTbl(&syncTables, "Database::retireRecords(2)");
syncTbl.lock(Shared);
- uint64 spaceToRetire = recordDataPool->activeMemory - recordScavengeFloor;
+ uint64 spaceToRetire = recordMemoryControl->getCurrentMemory(MemMgrRecordData) - recordScavengeFloor;
recordScavenge->computeThreshold(spaceToRetire);
for (Table *table = tableList; table; table = table->next)
@@ -1956,7 +1964,7 @@ void Database::scavengerThreadMain(void)
{
scavenge((scavengeForced > 0));
- if (recordDataPool->activeMemory < recordScavengeThreshold)
+ if (recordMemoryControl->getCurrentMemory(MemMgrRecordData) < recordScavengeThreshold)
{
INTERLOCKED_INCREMENT(scavengerThreadSleeping);
thread->sleep();
@@ -2611,32 +2619,32 @@ void Database::checkRecordScavenge(void)
syncMem.lock(Exclusive);
if ( !scavengerThreadSignaled
- && (recordDataPool->activeMemory > lastActiveMemoryChecked))
+ && (recordMemoryControl->getCurrentMemory(MemMgrRecordData) > lastActiveMemoryChecked))
{
// Start a new age generation regularly. Note that since activeMemory
// can go down due to a recent scavenge, it is possible for
- // lastGenerationMemory to be > recordDataPool->activeMemory
+ // lastGenerationMemory to be > recordMemoryControl->getCurrentMemory()
- if ( (int64) (recordDataPool->activeMemory - lastGenerationMemory)
+ if ( (int64) (recordMemoryControl->getCurrentMemory(MemMgrRecordData) - lastGenerationMemory)
> (int64) recordScavengeMaxGroupSize)
{
// Let the scavenger run to prune records.
// It will also retire records if recordScavengeThreshold has been reached.
INTERLOCKED_INCREMENT (currentGeneration);
- lastGenerationMemory = recordDataPool->activeMemory;
+ lastGenerationMemory = recordMemoryControl->getCurrentMemory(MemMgrRecordData);
INTERLOCKED_INCREMENT(scavengerThreadSignaled);
scavengerThreadWakeup();
}
- else if (recordDataPool->activeMemory >= recordScavengeThreshold)
+ else if (recordMemoryControl->getCurrentMemory(MemMgrRecordData) >= recordScavengeThreshold)
{
INTERLOCKED_INCREMENT(scavengerThreadSignaled);
scavengerThreadWakeup();
}
- lastActiveMemoryChecked = recordDataPool->activeMemory;
+ lastActiveMemoryChecked = recordMemoryControl->getCurrentMemory(MemMgrRecordData);
}
}
}
@@ -2757,7 +2765,7 @@ void Database::flushWait(void)
cache->flushWait();
}
-void Database::setLowMemory(void)
+void Database::setLowMemory(uint64 spaceNeeded)
{
if (!backLog)
{
@@ -2769,9 +2777,12 @@ void Database::setLowMemory(void)
}
lowMemory = true;
+ lowMemoryCount = MAX_LOW_MEMORY;
+// this->transactionManager->setLowMemory(spaceNeeded);
}
void Database::clearLowMemory(void)
{
+// this->transactionManager->clearLowMemory();
lowMemory = false;
}
=== modified file 'storage/falcon/Database.h'
--- a/storage/falcon/Database.h 2009-02-26 20:04:31 +0000
+++ b/storage/falcon/Database.h 2009-03-03 07:09:29 +0000
@@ -1,4 +1,4 @@
-/* Copyright � 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
+/* Copyright (C) 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -61,7 +61,7 @@ static const int OUT_OF_RECORD_MEMORY_RE
// Milliseconds per iteration to wait for the Scavenger
-static const int SCAVENGE_WAIT_MS = 20;
+static const int SCAVENGE_WAIT_MS = 50;//500;
// Scavenger cycles per call to updateCardinalities()
@@ -120,6 +120,7 @@ class IndexKey;
class InfoTable;
class TableSpace;
class MemMgr;
+class MemControl;
class RecordScavenge;
class PriorityScheduler;
class SQLException;
@@ -255,7 +256,7 @@ public:
void setIOError(SQLException* exception);
void clearIOError(void);
void flushWait(void);
- void setLowMemory(void);
+ void setLowMemory(uint64 spaceNeeded);
void clearLowMemory(void);
@@ -332,7 +333,10 @@ public:
volatile INTERLOCK_TYPE scavengeForced;
PageWriter *pageWriter;
PreparedStatement *updateCardinality;
- MemMgr *recordDataPool;
+ MemControl *recordMemoryControl;
+ MemMgr *recordDataPool; // Record data pool (no object metadata)
+ MemMgr *recordPool; // Record object pool
+ MemMgr *recordVersionPool; // RecordVersion object pool
time_t startTime;
volatile int deltaTime;
@@ -356,6 +360,7 @@ public:
uint64 lastGenerationMemory;
uint64 lastActiveMemoryChecked;
uint64 scavengeCount;
+ uint64 lowMemoryCount;
time_t creationTime;
volatile time_t lastScavenge;
};
=== modified file 'storage/falcon/MemControl.cpp'
--- a/storage/falcon/MemControl.cpp 2008-05-14 18:39:57 +0000
+++ b/storage/falcon/MemControl.cpp 2009-03-03 07:09:29 +0000
@@ -20,20 +20,53 @@
MemControl::MemControl(void)
{
count = 0;
+ maxMemory = 0;
}
MemControl::~MemControl(void)
{
}
+// Verify that the requested memory will not exceed the memory limit.
+//
+// If a limit has not been set for the group of memory pools, compare
+// against the total of individual pool limits.
+//
+// Note that allocation may occur during static initialization,
+// before the MemControl has been fully initialized and the memory
+// limits set.
+
bool MemControl::poolExtensionCheck(uint size)
{
uint64 inUse = size;
- for (int n = 0; n < count; ++n)
- inUse += pools[n]->currentMemory;
+ // If non-zero, MemControl::maxMemory supercedes the
+ // total of the individual pool limits.
+
+ if (maxMemory)
+ {
+ for (int n = 0; n < count; ++n)
+ inUse += pools[n]->currentMemory;
+
+ return inUse < maxMemory;
+ }
+ else
+ {
+ // If no group maximum is set, total the individual pool
+ // limits. Only consider pools for which a limit has been set.
+
+ uint64 poolMax = 0;
+ for (int n = 0; n < count; ++n)
+ {
+ if (pools[n]->maxMemory)
+ {
+ inUse += pools[n]->currentMemory;
+ poolMax += pools[n]->maxMemory;
+ }
+ }
- return inUse < maxMemory;
+ return poolMax ? (inUse < poolMax) : true;
+ }
}
void MemControl::addPool(MemMgr* pool)
@@ -42,7 +75,48 @@ void MemControl::addPool(MemMgr* pool)
pool->memControl = this;
}
+// Set the memory limit for the group of pools
+
void MemControl::setMaxSize(uint64 size)
{
maxMemory = size;
}
+
+// Set the memory limit for a specific pool. Note that the
+// size is only set for the first pool matching mgrId.
+
+void MemControl::setMaxSize(int mgrId, uint64 size)
+{
+ uint64 groupTotal = 0;
+
+ for (int n = 0; n < count; ++n)
+ groupTotal += pools[n]->maxMemory;
+
+ // Set the memory limit after checking for overflow
+
+ for (int n = 0; n < count; ++n)
+ {
+ if (pools[n]->id == mgrId)
+ {
+ uint64 remaining = MaxTotalMemory - groupTotal;
+ pools[n]->maxMemory = (size < remaining ? size : remaining);
+ break;
+ }
+ }
+}
+
+// Total memory in use for the pools in this group,
+// specified by poolMask. Default is all pools.
+
+uint64 MemControl::getCurrentMemory(int poolMask)
+{
+ uint64 inUse = 0;
+
+ for (int n = 0; n < count; ++n)
+ if (pools[n]->id & poolMask)
+ inUse += pools[n]->currentMemory;
+
+ return inUse;
+}
+
+
=== modified file 'storage/falcon/MemControl.h'
--- a/storage/falcon/MemControl.h 2008-05-14 18:39:57 +0000
+++ b/storage/falcon/MemControl.h 2009-03-03 07:09:29 +0000
@@ -29,8 +29,10 @@ public:
int count;
uint64 maxMemory;
virtual bool poolExtensionCheck(uint size);
+ virtual uint64 getCurrentMemory(int poolMask = MemMgrAllPools);
void addPool(MemMgr* pool);
void setMaxSize(uint64 size);
+ void setMaxSize(int mgrId, uint64 size);
};
#endif
=== modified file 'storage/falcon/MemMgr.cpp'
--- a/storage/falcon/MemMgr.cpp 2009-02-23 22:42:36 +0000
+++ b/storage/falcon/MemMgr.cpp 2009-03-03 07:09:29 +0000
@@ -35,6 +35,7 @@
#include "Stream.h"
#include "InfoTable.h"
#include "MemControl.h"
+#include "RecordVersion.h"
#ifdef HAVE_purify
#ifdef HAVE_CONFIG
@@ -79,10 +80,18 @@ const int validateMinutia = 16;
// Nominal memory limits at startup--final values set during initialization
bool memoryManagerAlive;
-static MemMgr memoryManager(defaultRounding, FREE_OBJECTS_SIZE, HEAP_SIZE,&memoryManagerAlive);
-static MemMgr recordManager(defaultRounding, 2, HEAP_SIZE);
-//static MemMgr recordObjectManager (defaultRounding, sizeof(RecordVersion) + 100, HEAP_SIZE);
-static MemControl memControl;
+static MemControl memControlGeneral;
+static MemControl memControlRecord;
+
+// General purpose memory
+static MemMgr memoryManager(MemMgrGeneral, defaultRounding, FREE_OBJECTS_SIZE, HEAP_SIZE,&memoryManagerAlive, &memControlGeneral);
+
+// Record data
+static MemMgr recordDataManager(MemMgrRecordData, defaultRounding, 2, HEAP_SIZE, NULL, &memControlRecord);
+
+// Record metadata: Record object and RecordVersion object
+static MemMgr recordManager(MemMgrRecord, defaultRounding, sizeof(RecordVersion) + 100, HEAP_SIZE, NULL, &memControlRecord);
+static MemMgr recordVersionManager(MemMgrRecordVersion, defaultRounding, sizeof(RecordVersion) + 100, HEAP_SIZE, NULL, &memControlRecord);
#ifdef _DEBUG
static void *stopAddress;
@@ -161,12 +170,12 @@ struct Client {
void* MemMgrRecordAllocate (size_t size, const char *file, int line)
{
- return recordManager.allocateDebug (size, file, line);
+ return recordDataManager.allocateDebug (size, file, line);
}
void MemMgrRecordDelete (char *record)
{
- recordManager.releaseDebug (record);
+ recordDataManager.releaseDebug (record);
}
#else
void* MemMgrPoolAllocate (MemMgr *pool, size_t s)
@@ -192,12 +201,12 @@ struct Client {
void* MemMgrRecordAllocate (size_t size, const char *file, int line)
{
- return recordManager.allocate (size);
+ return recordDataManager.allocate (size);
}
void MemMgrRecordDelete (char *record)
{
- recordManager.release (record);
+ recordDataManager.release (record);
}
#endif
@@ -205,8 +214,9 @@ struct Client {
void MemMgrValidate ()
{
memoryManager.validate();
+ recordDataManager.validate();
recordManager.validate();
- //recordObjectManager.validate();
+ recordVersionManager.validate();
}
void MemMgrValidate (void *object)
@@ -221,7 +231,7 @@ void MemMgrAnalyze(int mask, Stream *str
stream->putSegment ("Memory\n");
memoryManager.analyze (mask, stream, NULL, NULL);
stream->putSegment ("Records\n");
- recordManager.analyze (mask, stream, NULL, NULL);
+ recordDataManager.analyze (mask, stream, NULL, NULL);
//LEAVE_CRITICAL_SECTION;
}
@@ -238,47 +248,61 @@ void MemMgrAnalyze(MemMgrWhat what, Info
break;
case MemMgrRecordSummary:
+ recordDataManager.analyze(0, NULL, infoTable, NULL);
recordManager.analyze(0, NULL, infoTable, NULL);
- //recordObjectManager.analyze(0, NULL, infoTable, NULL);
+ recordVersionManager.analyze(0, NULL, infoTable, NULL);
break;
case MemMgrRecordDetail:
+ recordDataManager.analyze(0, NULL, NULL, infoTable);
recordManager.analyze(0, NULL, NULL, infoTable);
- //recordObjectManager.analyze(0, NULL, NULL, infoTable);
+ recordVersionManager.analyze(0, NULL, NULL, infoTable);
break;
}
}
+// Set memory limit for the record data pool
+
void MemMgrSetMaxRecordMember (long long size)
{
- if (!recordManager.memControl)
- {
- memControl.addPool(&recordManager);
- //memControl.addPool(&recordObjectManager);
- }
- memControl.setMaxSize(size);
+ memControlRecord.setMaxSize(MemMgrRecordData, size);
}
MemMgr* MemMgrGetFixedPool (int id)
{
switch (id)
{
- case MemMgrPoolGeneral:
+ case MemMgrGeneral:
return &memoryManager;
- case MemMgrPoolRecordData:
- return &recordManager;
+ case MemMgrRecordData:
+ return &recordDataManager;
- /***
- case MemMgrPoolRecordObject:
- return &recordObjectManager;
- ***/
+ case MemMgrRecord:
+ return &recordManager;
+ case MemMgrRecordVersion:
+ return &recordVersionManager;
+
default:
return NULL;
}
}
+MemControl* MemMgrGetControl (int id)
+{
+ switch (id)
+ {
+ case MemMgrControlGeneral:
+ return &memControlGeneral;
+
+ case MemMgrControlRecord:
+ return &memControlRecord;
+
+ default:
+ return NULL;
+ }
+}
void MemMgrLogDump()
{
@@ -288,23 +312,24 @@ void MemMgrLogDump()
#endif
}
-
-MemMgr::MemMgr(int rounding, int cutoff, int minAlloc, bool *alive) : mutex("MemMgr::mutex")
-{
- signature = defaultSignature;
- roundingSize = rounding;
- threshold = cutoff;
- minAllocation = minAlloc;
- currentMemory = 0;
- blocksAllocated = 0;
- blocksActive = 0;
- activeMemory = 0;
- numberBigHunks = 0;
- numberSmallHunks = 0;
- bigHunks = NULL;
- smallHunks = NULL;
- memControl = NULL;
-
+MemMgr::MemMgr(int mgrId, int rounding, int cutoff, int minAlloc, bool *alive, MemControl *memCtrl)
+ : id(mgrId), roundingSize(rounding), threshold(cutoff), minAllocation(minAlloc),
+ memControl(memCtrl), mutex("MemMgr::mutex"), isAlive(alive)
+{
+ signature = defaultSignature;
+ activeMemory = 0;
+ currentMemory = 0;
+ maxMemory = 0;
+ blocksAllocated = 0;
+ blocksActive = 0;
+ numberBigHunks = 0;
+ numberSmallHunks = 0;
+ bigHunks = NULL;
+ smallHunks = NULL;
+
+ if (memControl)
+ memControl->addPool(this);
+
int vecSize = (cutoff + rounding) / rounding;
int l = vecSize * sizeof (void*);
@@ -313,14 +338,11 @@ MemMgr::MemMgr(int rounding, int cutoff,
//freeBlocks.nextLarger = freeBlocks.priorSmaller = &freeBlocks;
//freeBlockTree = NULL;
junk.larger = junk.smaller = &junk;
- isAlive = alive;
+
if(alive)
- {
*alive = true;
- }
}
-
MemMgr::MemMgr(void* arg1, void* arg2) : mutex("MemMgr::mutex")
{
MemMgr();
@@ -576,7 +598,6 @@ void* MemMgr::allocateDebug(size_t size,
return &memory->body;
}
-
void MemMgr::release(void* object)
{
if (object)
=== modified file 'storage/falcon/MemMgr.h'
--- a/storage/falcon/MemMgr.h 2009-01-08 09:05:26 +0000
+++ b/storage/falcon/MemMgr.h 2009-03-03 07:09:29 +0000
@@ -35,6 +35,10 @@ static const int defaultCutoff = 4096;
static const int defaultAllocation = 65536;
static const int defaultSignature = 12345678;
+// Limit of the total memory for the pools within a MemControl group
+
+static const uint64 MaxTotalMemory = 0xffffffffffffffffLL;
+
class MemMgr;
class Stream;
class InfoTable;
@@ -107,11 +111,17 @@ public:
class MemMgr
{
public:
- MemMgr(int rounding=defaultRounding, int cutoff=defaultCutoff,
- int minAllocation=defaultAllocation, bool *alive = NULL);
+ MemMgr( int mgrId = MemMgrDefault,
+ int rounding = defaultRounding,
+ int cutoff = defaultCutoff,
+ int minAllocation = defaultAllocation,
+ bool *alive = NULL,
+ MemControl *memCtrl = NULL);
+
MemMgr(void* arg1, void* arg2);
virtual ~MemMgr(void);
+ int id;
int signature;
int roundingSize;
int threshold;
@@ -129,6 +139,7 @@ public:
Mutex mutex; // Win32 critical regions are faster than SyncObject
int64 currentMemory;
uint64 activeMemory;
+ uint64 maxMemory;
int blocksAllocated;
int blocksActive;
bool *isAlive;
=== modified file 'storage/falcon/MemoryManager.h'
--- a/storage/falcon/MemoryManager.h 2008-10-31 15:42:42 +0000
+++ b/storage/falcon/MemoryManager.h 2009-03-03 07:09:29 +0000
@@ -40,6 +40,7 @@
class Stream;
class InfoTable;
class MemMgr;
+class MemControl;
struct MemObject;
#ifdef _DEBUG
@@ -110,9 +111,19 @@ enum MemMgrWhat {
MemMgrRecordDetail
};
-static const int MemMgrPoolGeneral = 0;
-static const int MemMgrPoolRecordData = 1;
-static const int MemMgrPoolRecordObject = 2;
+// Memory pool identifiers
+
+static const int MemMgrDefault = 1;
+static const int MemMgrGeneral = 2;
+static const int MemMgrRecordData = 4;
+static const int MemMgrRecord = 8;
+static const int MemMgrRecordVersion = 16;
+static const int MemMgrAllPools = -1;
+
+// Memory pool group identifiers
+
+static const int MemMgrControlGeneral = 0; // general memory pool
+static const int MemMgrControlRecord = 1; // record and object memory pools
extern void MemMgrAnalyze(MemMgrWhat what, InfoTable *table);
extern void MemMgrRelease (void *object);
@@ -122,6 +133,7 @@ extern void* MemMgrRecordAllocate (size_
extern void MemMgrRecordDelete (char *record);
extern void MemMgrSetMaxRecordMember (long long size);
extern MemMgr* MemMgrGetFixedPool (int id);
+extern MemControl* MemMgrGetControl (int id);
extern MemObject* MemMgrFindPriorBlock (void *block);
=== modified file 'storage/falcon/RecordScavenge.cpp'
--- a/storage/falcon/RecordScavenge.cpp 2009-02-25 16:39:29 +0000
+++ b/storage/falcon/RecordScavenge.cpp 2009-03-03 07:09:29 +0000
@@ -21,6 +21,7 @@
#include "RecordVersion.h"
#include "Log.h"
#include "MemMgr.h"
+#include "MemControl.h"
#include "Sync.h"
#include "Transaction.h"
#include "TransactionManager.h"
@@ -34,7 +35,7 @@ RecordScavenge::RecordScavenge(Database
veryOldRecords = 0;
veryOldRecordSpace = 0;
- startingActiveMemory = database->recordDataPool->activeMemory;
+ startingActiveMemory = database->recordMemoryControl->getCurrentMemory(MemMgrRecordData);
prunedActiveMemory = 0;
retiredActiveMemory = 0;
@@ -239,7 +240,12 @@ Record* RecordScavenge::inventoryRecord(
uint64 RecordScavenge::computeThreshold(uint64 spaceToRetire)
{
uint64 totalSpace = veryOldRecordSpace;
- scavengeGeneration = 0;
+ scavengeGeneration = baseGeneration;
+
+ // Don't mess around if memory is critical
+
+ if (database->lowMemory)
+ return scavengeGeneration;
// The baseGeneration is the currentGeneration when the scavenge started
// It is in ageGroups[0]. Next oldest in ageGroups[1], etc.
@@ -254,10 +260,7 @@ uint64 RecordScavenge::computeThreshold(
scavengeGeneration = baseGeneration - n;
}
- // We still may want to scavenge even if the age group total is
- // too small, so use the base generation as a starting point.
-
- return (scavengeGeneration ? scavengeGeneration : baseGeneration);
+ return scavengeGeneration;
}
void RecordScavenge::print(void)
=== modified file 'storage/falcon/RecordVersion.cpp'
--- a/storage/falcon/RecordVersion.cpp 2009-02-26 20:04:31 +0000
+++ b/storage/falcon/RecordVersion.cpp 2009-03-03 07:09:29 +0000
@@ -1,4 +1,4 @@
-/* Copyright � 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
+/* Copyright (C) 2006-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -68,6 +68,9 @@ RecordVersion::RecordVersion(Table *tbl,
RecordVersion::RecordVersion(Database* database, Serialize *stream) : Record(database, stream)
{
+ // Reconstitute a record version and recursively restore all
+ // prior versions from 'stream'
+
virtualOffset = stream->getInt64();
transactionId = stream->getInt();
int priorType = stream->getInt();
@@ -447,6 +450,8 @@ void RecordVersion::serialize(Serialize*
stream->putInt64(virtualOffset);
stream->putInt(transactionId);
+ // Recursively serialize the prior version chain
+
if (priorVersion)
{
stream->putInt(priorVersion->isVersion());
=== modified file 'storage/falcon/Scavenger.cpp'
--- a/storage/falcon/Scavenger.cpp 2009-01-08 09:05:26 +0000
+++ b/storage/falcon/Scavenger.cpp 2009-03-03 07:09:29 +0000
@@ -65,7 +65,8 @@ void Scavenger::scavenge()
void Scavenger::execute(Scheduler * scheduler)
{
- database->signalScavenger();
+ bool forced = database->scavengeForced || database->lowMemory;
+ database->signalScavenger(forced);
getNextEvent();
scheduler->addEvent (this);
}
=== modified file 'storage/falcon/Table.cpp'
--- a/storage/falcon/Table.cpp 2009-03-03 06:30:21 +0000
+++ b/storage/falcon/Table.cpp 2009-03-03 07:09:29 +0000
@@ -3669,7 +3669,7 @@ RecordVersion* Table::allocRecordVersion
if ((++database->recordPoolAllocCount & 0x7F) == 0)
database->checkRecordScavenge();
- return POOL_NEW(database->recordDataPool) RecordVersion(this, format, transaction, priorVersion);
+ return POOL_NEW(database->recordVersionPool) RecordVersion(this, format, transaction, priorVersion);
}
catch (SQLException& exception)
@@ -3706,7 +3706,7 @@ Record* Table::allocRecord(int recordNum
if ((++database->recordPoolAllocCount & 0x7F) == 0)
database->checkRecordScavenge();
- return POOL_NEW(database->recordDataPool) Record (this, recordNumber, stream);
+ return POOL_NEW(database->recordPool) Record (this, recordNumber, stream);
}
catch (SQLException& exception)
| Thread |
|---|
| • bzr commit into mysql-6.0-falcon-team branch (christopher.powers:3046)Bug#32838 Bug#33177 Bug#42651 | Christopher Powers | 3 Mar |
