List:Falcon Storage Engine« Previous MessageNext Message »
From:Christopher Powers Date:February 13 2009 7:41pm
Subject:Re: Falcon Backlogging
View as plain text  
Thanks, Jim. This is a very good start. I'll give more thought to this 
over the weekend.

Kevin won't be able to make the 3:30pm call, so let's reschedule for 
next week.

Is Monday morning ok, or would you prefer Tuesday?

Jim Starkey wrote:
> Here are some preliminary thoughts on backlogging.
> 
> First, backlogging chilled records was intended as a starting point, not 
> the full solution.  Since it doesn't start until memory low as been set, 
> it can't be the full solution.
> 
> The questions are "what to backlog", "when to backlog", and who should 
> do it.
> 
> I think the "who" question is obvious -- the scavenger.  When memory is 
> low and it sees a backlog candidate (whatever that means), and the use 
> count of of the record is one, it should probably backlog it.
> 
> The "what to backlog" question is complicated.  Committed records with 
> back versions seem like obvious candidates (older ones are better 
> candidates than newer ones).  Active but chilled records are also 
> candidates, but the interaction with transaction savepoints may be 
> tricky.  Anything in use (useCount > 1) are not candidates.  Active 
> records that haven't been chilled as poor candidates since they need to 
> be reconstituted to be chilled or committed, so the memory gain is 
> transient and the overhead relatively high.
> 
> For better or worse, Table::backlogRecord operates with an exclusive 
> lock on the record tree, so it is possible to make most record status 
> decisions safely.
> 
> One last thing to keep in mind is that backlogging every possible record 
> isn't necessary, desirable, or even possible.  The goal is free up 
> enough working memory to continue operations.  There is always a reason 
> that the record cache is full up of record versions.  Addressing those 
> reasons is all that is necessary.  A full general solution probably 
> isn't necessary.
> 
Thread
Falcon BackloggingChristopher Powers13 Feb
  • Re: Falcon BackloggingJim Starkey13 Feb
    • Re: Falcon BackloggingChristopher Powers13 Feb