Here are some preliminary thoughts on backlogging.
First, backlogging chilled records was intended as a starting point, not
the full solution. Since it doesn't start until memory low as been set,
it can't be the full solution.
The questions are "what to backlog", "when to backlog", and who should
do it.
I think the "who" question is obvious -- the scavenger. When memory is
low and it sees a backlog candidate (whatever that means), and the use
count of of the record is one, it should probably backlog it.
The "what to backlog" question is complicated. Committed records with
back versions seem like obvious candidates (older ones are better
candidates than newer ones). Active but chilled records are also
candidates, but the interaction with transaction savepoints may be
tricky. Anything in use (useCount > 1) are not candidates. Active
records that haven't been chilled as poor candidates since they need to
be reconstituted to be chilled or committed, so the memory gain is
transient and the overhead relatively high.
For better or worse, Table::backlogRecord operates with an exclusive
lock on the record tree, so it is possible to make most record status
decisions safely.
One last thing to keep in mind is that backlogging every possible record
isn't necessary, desirable, or even possible. The goal is free up
enough working memory to continue operations. There is always a reason
that the record cache is full up of record versions. Addressing those
reasons is all that is necessary. A full general solution probably
isn't necessary.
--
Jim Starkey
President, NimbusDB, Inc.
978 526-1376
