From: Christopher Powers
Date: January 15 2009 1:56am
Subject: Re: More on Scavenge vs DDL
List-Archive: http://lists.mysql.com/falcon/373
Message-Id: <496E97C9.60105@sun.com>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed; charset=ISO-8859-1
Content-Transfer-Encoding: 7BIT

Solution below.

Christopher Powers wrote:
> I decoupled updateCardinalities() from the Scavenger and moved it on to 
> its own thread. Now, each scavenge simply signals it and moves on, no 
> waiting on syncSysDDL.
> 
> BUT...
> 
> Database::scavengeRecords() commits pending system transactions, which 
> of course requires a lock syncSysDDL. In the old code, this was done 
> only inside of syncScavenge AND if the scavenge was not 'forced'.
> 
> Load-based scavenges are the current equivalent of forced scavenges, and 
> also should not commit pending system transactions.

Kevin, here's what works (see code below):

1) Database::scavengerThreadMain()sets 'signaled' for unscheduled scavenges.
2) At thread wakeup (signaled or scheduled), pass 'signaled' to 
Database::scavenge().
3) Database::scavenge() only commits system transactions for scheduled 
scavenges.

With this change, a 10m row load and online create index runs like a champ.

Remaining issue: Scheduled scavenges can still block on syncSysDDL 
during Table::populateIndex() and prevent scavenge requests from 
memory-starved threads.

If we disable scheduled scavenges during memory-critical operations, 
then the problem goes away.

What do you think?


Code changes:


1) Set 'signaled' for non-scheduled scavenges:

void Database::scavengerThreadMain(void)
{
>>> bool signaled = false;  <<<

    Thread *thread = Thread::getThread("Database::scavengerThreadMain");
    thread->sleep(1000);

    while (!thread->shutdownInProgress)
       {
>>>   scavenge(signaled);  <<<
       signaled = false;
		
       if (recordDataPool->activeMemory < recordScavengeThreshold)
          {
          INTERLOCKED_INCREMENT(scavengerThreadSleeping);
          thread->sleep();
>>>      signaled = scavengerThreadSignaled != 0;  <<<
          scavengerThreadSignaled = 0;
          INTERLOCKED_DECREMENT(scavengerThreadSleeping);
          }
       }
[...]


2) Only commit pending system transactions for non-scheduled transactions:

void Database::scavengeRecords(bool signaled) <<<
{
     // Commit pending system transactions before proceeding

     if (!signaled && systemConnection->transaction)   <<<
        commitSystemTransaction();




> 
> 
> // NEW CODE
> 
> void Database::scavengeRecords(void)
> {
>    // Commit pending system transactions before proceeding
> 
>    if (systemConnection->transaction)
>       commitSystemTransaction();
> 
>    Sync syncScavenger(&syncScavenge, 
> "Database::scavengeRecords(Scavenge)");
>    syncScavenger.lock(Exclusive)
> [...]
> 
> 
> // OLD CODE
> 
> void Database::retireRecords(bool forced)
> {
>     int cycle = scavengeCycle;
>     
>     Sync syncScavenger(&syncScavenge, "Database::retireRecords(1)");
>     syncScavenger.lock(Exclusive);
> 
>     if (forced && scavengeCycle > cycle)
>        return;
>     
>     // Commit pending system transactions before proceeding
>     
>     if (!forced && systemConnection->transaction)
>        commitSystemTransaction();
>