List:Falcon Storage Engine« Previous MessageNext Message »
From:Christopher Powers Date:January 15 2009 2:16am
Subject:Re: More on Scavenge vs DDL
View as plain text  
Vladislav Vaintroub wrote:
> Chris, just out of curiosity - is it necessary to have a thread for each
> single task? I might be mistaken, but do not  we have a timer or similar for
> that, specifically for periodic tasks?

Yes, we have two timing threads, the Scheduler and the Ticker. The 
Scheduler runs tasks at predefined fixed-length intervals. The Ticker 
runs them every second.

I initially proposed scheduling updateCardinalities() as a separate 
task, like the Scavenger. Kevin suggested that it would be better to 
signal updateCardinalities *from* the Scavenger, which is now more 
responsive to cache activity.

Cardinality has historically been updated at scavenge time because 
that's a naturally good time to do it--both tasks process the entire 
record tree.

This approach has three benefits:

1) Scavenger and updateCardinalities() are independent (no syncSysDDL 
nonesense).
2) Cardinality will more closely reflect reality because it is invoked 
at scavenge time.
3) We retain the intent of the original design.


> 
>> -----Original Message-----
>> From: Christopher.Powers@stripped [mailto:Christopher.Powers@stripped]
>> Sent: Thursday, January 15, 2009 2:04 AM
>> To: Kevin Lewis
>> Cc: FalconDev
>> Subject: More on Scavenge vs DDL
>>
>> I decoupled updateCardinalities() from the Scavenger and moved it on to
>> its own thread. Now, each scavenge simply signals it and moves on, no
>> waiting on syncSysDDL.
>>
>> BUT...
>>
>> Database::scavengeRecords() commits pending system transactions, which
>> of course requires a lock syncSysDDL. In the old code, this was done
>> only inside of syncScavenge AND if the scavenge was not 'forced'.
>>
>> Load-based scavenges are the current equivalent of forced scavenges,
>> and
>> also should not commit pending system transactions.
>>
>>
>> // NEW CODE
>>
>> void Database::scavengeRecords(void)
>> {
>>     // Commit pending system transactions before proceeding
>>
>>     if (systemConnection->transaction)
>>        commitSystemTransaction();
>>
>>     Sync syncScavenger(&syncScavenge,
>> "Database::scavengeRecords(Scavenge)");
>>     syncScavenger.lock(Exclusive)
>> [...]
>>
>>
>> // OLD CODE
>>
>> void Database::retireRecords(bool forced)
>> {
>>      int cycle = scavengeCycle;
>>
>>      Sync syncScavenger(&syncScavenge, "Database::retireRecords(1)");
>>      syncScavenger.lock(Exclusive);
>>
>>      if (forced && scavengeCycle > cycle)
>>         return;
>>
>>      // Commit pending system transactions before proceeding
>>
>>      if (!forced && systemConnection->transaction)
>>         commitSystemTransaction();
>>
>> --
>> Falcon Storage Engine Mailing List
>> For list archives: http://lists.mysql.com/falcon
>> To unsubscribe:    http://lists.mysql.com/falcon?unsub=1
> 
> 
Thread
More on Scavenge vs DDLChristopher Powers15 Jan
  • RE: More on Scavenge vs DDLVladislav Vaintroub15 Jan
    • Re: More on Scavenge vs DDLChristopher Powers15 Jan
      • RE: More on Scavenge vs DDLVladislav Vaintroub15 Jan
        • Re: More on Scavenge vs DDLChristopher Powers15 Jan
  • Re: More on Scavenge vs DDLChristopher Powers15 Jan