From: Jim Starkey
Date: February 13 2009 5:22pm
Subject: Re: Falcon Backlogging
List-Archive: http://lists.mysql.com/falcon/514
Message-Id: <4995AC5C.10408@nimbusdb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Here are some preliminary thoughts on backlogging.

First, backlogging chilled records was intended as a starting point, not 
the full solution.  Since it doesn't start until memory low as been set, 
it can't be the full solution.

The questions are "what to backlog", "when to backlog", and who should 
do it.

I think the "who" question is obvious -- the scavenger.  When memory is 
low and it sees a backlog candidate (whatever that means), and the use 
count of of the record is one, it should probably backlog it.

The "what to backlog" question is complicated.  Committed records with 
back versions seem like obvious candidates (older ones are better 
candidates than newer ones).  Active but chilled records are also 
candidates, but the interaction with transaction savepoints may be 
tricky.  Anything in use (useCount > 1) are not candidates.  Active 
records that haven't been chilled as poor candidates since they need to 
be reconstituted to be chilled or committed, so the memory gain is 
transient and the overhead relatively high.

For better or worse, Table::backlogRecord operates with an exclusive 
lock on the record tree, so it is possible to make most record status 
decisions safely.

One last thing to keep in mind is that backlogging every possible record 
isn't necessary, desirable, or even possible.  The goal is free up 
enough working memory to continue operations.  There is always a reason 
that the record cache is full up of record versions.  Addressing those 
reasons is all that is necessary.  A full general solution probably 
isn't necessary.

-- 
Jim Starkey
President, NimbusDB, Inc.
978 526-1376