Jim, Kevin, everybody,
I am working on a group of bugs related to transactions not returning
the correct data every time. These issues are most likely related to our
implementation of isolation levels and visibility of record versions. I
have found the cause for one of these bugs but would like input on what
is the best way to solve it.
The bug I have been analyzing is Bug #41357 "falcon.falcon_bug_34351_C
fails with assertion IS_CONSISTENT_READ". The assert happens in
Table::fetchForUpdate() due to the call to
Transaction::getRelativeState() returns CommittedInvisible for a
recordVersion when the current
transaction is running in isolation level TRANSACTION_WRITE_COMMITTED.
This return value
should only happen if the isolation level is TRANSACTION_CONSISTENT_READ.
After having reproduced this assert several times with instrumentation
in the source it
seems like this problems is caused by that the Transaction object that
the recordVersion
object has a pointer to and which is used for computing the relative
state between the two
transaction "changes state" during the execution of
Transaction::getRelativeState().
Here is what normally happens when the CommittedInvisible state is
reached by the following code path:
1. Transaction::getRelativeState() is called and the following code is run:
if (transaction->state == Committed)
{
// Return CommittedVisible if the other trans has a lower TransId and
// it was committed when we started.
if (visible (transaction, transId, FOR_WRITING))
return CommittedVisible;
return CommittedInvisible;
}
when this code is run the transaction->state == Committed (and the if
test is true).
This results in Transaction::visible() being called.
2. In the code for Transaction::visible() the following code is run:
// If the other transaction is not yet committed, the trans is not visible.
if (transaction->state != Committed)
return false;
When the same test now is evaluated "transaction->state != Committed"
this test too
succeed and returns that the transaction should not be visible.
So based on traces in the code and studying core files it seems like the
transaction->state variable changes value from being Committed to not
being Committed
during the execution of these two methods. This leads to
Transaction::visible() returning "false" which again leads to
Transaction::getRelativeState() returning CommittedInvisible (and the
assert occurs).
The reason for the "mysterious" state change (at least in some of the
crashes I have reproduced) is that the transaction object which the
record version points too is deleted by another thread and thus the
check for committed (or not) in Transaction::visible() is now checking
"free memory".
This situation seems very similar to an issue Kevin had (and still
have?) with the scavenger checking which records that can be scavenged.
Questions:
-how do we either handle that Transaction object that is being pointed
to by RecordVersion objects are deleted at "any time". Like the code for
Transaction::getRelativeState and Transaction::visible() is now this can
lead to wrong conclusions about which records are visible or not.
-or do we need to introduce some kind of locking so that we can safely
use RecordVersion objects and the Transaction object they normally
refers to?
-or can we do something about the order we clean up old transaction
objects versus the record version objects to avoid that we have record
object without a transaction object?
I do not know enough about this part of the code (yet) to propose a good
solution so any suggestions would be appreciated.
Olav
