List:Falcon Storage Engine« Previous MessageNext Message »
From:Kevin Lewis Date:March 16 2009 6:01pm
Subject:Re: New Transaction State object (Was: Problems with record visibility
and how it is computed)
View as plain text  
The main reason this solution is preferable to a CycleManager solution 
is because we can reduce the size of the Recordversion object.  I 
definitely support moving Recordversion::transactionId to 
TransactionState::transactionId.

More comments below;

Olav Sandstaa wrote:
> Subj: New TransactionState object
> 
> Hi,
> 
> In our Athens meeting I went through the alternatives for how to solve 
> the issues with the "RecordVersion to Transaction object pointer". We 
> agreed on that introducing a "transaction state object" (proposal 9 
> quoted below) seemed to be the best and least intrusive alternative.
> 
> I have not looked into the actual existing code yet but here is an 
> initial overview of how my current plan for the "transaction state 
> object" is (it might changes as I look more at the existing code and use 
> of it):
> 
> A. New data structure:
> ======================
> 
>   A new class called TransactionState will be introduced. This will as a 
> minimum store the following members:
> 
>   class TransactionState
>   {
>     volatile INTERLOCKED useCount;      }
       TransId transactionId;
 >     TransId commitId;

// also, either
       Transaction *transaction;

// or else the stuff from a transaction that a Recordversion may need, 
including;

       SyncObject syncIsActive;  // locked while transaction is active
       volatile INTERLOCK_TYPE state;

// Maybe more

>   It is also likely that we will move the transaction id to this
>   class (could save us from storing this in each RecordVersion
>   object). The transaction state of the transaction might also be
>   moved to this object.

Agree.  I think we should look at what is associated with the lifecycle, 
birth to death, of a transaction.  And what is associated with the 
reusable part of a transaction object.  We need to think about Jim's 
idea of whether we need two lists.  That said, it may be unnecessary to 
do this extra re-organizing for stability.  Having one list and re-using 
Transaction objects after non-read-only commits may be only a 
performance issue.  And right now, we are all about stability.

> 
>   Changes to existing classes:
> 
>   1. The Transaction object will have a pointer to a TransactionState
>      object. When committing the transaction the commitId will be
>      stored in the TransactionState object, not in the transaction
>      object.
> 
>   2. The RecordVersion object will have a pointer to a
>      TransactionState object (replacing the existing pointer to the
>      Transaction object) (if there is need for accessing the
>      Transaction object from the RecordVersion object we will either
>      keep the transaction pointer or have a transaction pointer from
>      the TransactionState object back to its Transaction object)

-   Transaction *transaction;
+   TransactionState *transactionState;

Note that one common use of the pointer from a Record to the Transaction 
is waitForTransaction.  That is why I mentioned the possibility of 
moving syncIsActive to TransactionState (as Jim suggested yesterday).


>   Goal with this new TransactionState object is to let it "live" as
>   long as there is RecordVersion objects refering to it (and let the
>   "real" transaction object be "purged" at "any time").

Perged or reused, whichever is eventually implemented.

However, we need to make sure that threads having a pointer to the 
Transaction object directly are all gone before it is purged.  Either we 
use a CycleManager for that or we assure that useCounts are used for 
these other pointers.

> 
> B. Controlling the "life span" of TransactionState objects
> ==========================================================
> 
>   We will use a reference counter in the Transaction state object for
>   controlling when a Transaction state object can be deleted.
>   Operations on this reference counter will be done by using
>   interlocked operations (if would probably be possible to implement
>   this without interlocked instruction but that would make the
>   implementation a bit more complex and harder to maintain and the
>   discussion in Athens concluded that the cost of using interlocked
>   operations for this case should not be a performance problem).
> 
>   The reference counter will be incremented the following places:
> 
>    1. In the Transcation's constructor when the TransactionState
>       object is created (we can also delay this until the first
>       Record is attached to a Transaction (Transaction::addRecord())
>       but it is simpler to just create it when we create the
>       Transaction object)

It can be created in Transaction::initialize().  The useCount from the 
TransactionState constructor would be applied to 
Transaction::transactionState.

>    2. For every RecordVersion having a pointer to it (most likely in
>       the RecordVersion's constructor - have not looked at the code
>       yet)
> 
>   The reference counter will be decremented the following places:
> 
>    1. In the Transaction's destructor
>    2. When the RecordVersion is deleted (most likely in the
>       destructor)

Yes, the destructor.

> 
>   There should be no objects using the TransactionState object
>   without having incremented the reference counter (or accessing it
>   indirectly via the a Transaction object or a RecordVersion object).
> 
> C. Usage of the TransactionState object:
> ========================================
> 
>   The initial usage of the TransactionState objects will be as
>   follows:
> 
>    1. Each time when creating a transaction object, we create a
>       transaction state object and a pointer to the transaction state
>       object from the transaction object.
> 
>    2. When we commit: the commitId is written to the transaction
>       state object (not the transaction object). So the commit is the
>       update to the transaction state object (not that it is possible
>       that we also will store the state of the Transaction in the
>       TransactionState object)
> 
>    3. Each time we create a record version object it has a pointer to
>       the transaction state object - and the reference count is
>       incremented (if needed also a pointer to the
>       transaction object (hopefully not, alternatively have a pointer
>       from the TransactionState object back to the transaction)
> 
>    4. When we need to compute visibility of records we only need to
>       use the transaction state object. This will have at least the
>       same life span as the record version objects pointing to it
>       (thus the pointer is guaranteed to be valid)
> 
>    5. When the scavenger removes a recordversion it decrement the
>       reference counter in the transaction state object
> 
>    6. When we purge old Transaction objects we decrement the reference
>       counter in the Transaction State object
> 
>    7. When the reference counter reaches 0 we delete the transaction
>       state object (by "self destruction")
> 
> 
> That is the current plan - and as most plans it might change :-)
> 
> Please let me know if you have any feedback or suggestions.
> 
> Olav
> 
> Olav Sandstaa wrote:
>> In addition to these I also have two new possible solutions:
>>
>> 9. (This is partly based on something Ann wrote: Make the transasction 
>> pointer indirect combined with my original proposal number 5: let the 
>> transaction objects live as long as there are record version pointing 
>> to them): Introduce a new "Transaction state object" that contains as 
>> a minimum the "commitId" but most likely also the "transactionId" and 
>> "state":
>>
>>     1. Each time when creating a transaction object, we create a 
>> transaction state object and a pointer to the transaction state object 
>> from the transaction object.
>>     2. When we commit: the commitId is written to the transaction 
>> state object (not the transaction object). So the commit is the update 
>> to the transaction state object.
>>     3. Each time we create a record version object it has a pointer to 
>> the transaction state object (and if needed also to the transaction 
>> object)
>>     4. Each time we add a record to a transaction (ie. update a 
>> record) we increment a useCount in the transaction state object: This 
>> should not need any extra locking (I think)
>>     5. When we need to compute visibility of records we only need to 
>> use the transaction state object. This will have at least the same 
>> life span as the record version objects pointing to it (thus the 
>> pointer is guaranteed to be valid)
>>     6. When the scavenger removes a recordversion it decrement the 
>> useCount in the transaction state object (I think this can be done 
>> safely without locking given that we have one scavenger)
>>     7. When the useCount reaches 0 we delete the transaction state 
>> object.
>>
>>   Issue/question: is there any other places in the code that will 
>> access this transaction state object than by accessing it from the 
>> record version object?
>>
>>    I assume when normal transactions accesses record version objects 
>> they are "locked" and this will implicitely also lock the transaction 
>> state object when normal transactions access it?
> 
> 
Thread
Problems with record visibility and how it is computedOlav Sandstaa21 Jan
  • search for null values in indexed columnsVladislav Vaintroub21 Jan
    • Re: search for null values in indexed columnsLars-Erik Bjørk21 Jan
    • Re: search for null values in indexed columnsKevin Lewis21 Jan
      • RE: search for null values in indexed columnsVladislav Vaintroub21 Jan
        • Re: search for null values in indexed columnsKevin Lewis21 Jan
      • Re: search for null values in indexed columnsJames Day23 Jan
  • Re: Problems with record visibility and how it is computedOlav Sandstaa23 Jan
    • Re: Problems with record visibility and how it is computedJim Starkey23 Jan
      • Re: Problems with record visibility and how it is computedKevin Lewis28 Jan
        • Re: Problems with record visibility and how it is computedJim Starkey28 Jan
          • Re: Problems with record visibility and how it is computedKevin Lewis28 Jan
            • Cycle Locking (was Problems with record visibility and how it iscomputed)Jim Starkey28 Jan
            • Re: Problems with record visibility and how it is computedAnn W. Harrison28 Jan
              • Re: Problems with record visibility and how it is computedJim Starkey28 Jan
                • Re: Problems with record visibility and how it is computedAnn W. Harrison28 Jan
                  • Re: Problems with record visibility and how it is computedOlav Sandstaa26 Feb
                    • New Transaction State object (Was: Problems with record visibility andhow it is computed)Olav Sandstaa16 Mar
                      • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Kevin Lewis16 Mar
                      • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Jim Starkey16 Mar
                        • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Olav Sandstaa19 Mar
                          • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Jim Starkey19 Mar
                            • RE: New Transaction State object (Was: Problems with record visibilityand how it is computed)Vladislav Vaintroub20 Mar
                            • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Olav Sandstaa20 Mar
                              • Re: New Transaction State object (Was: Problems with record visibilityand how it is computed)Jim Starkey20 Mar
          • Re: Problems with record visibility and how it is computedOlav Sandstaa28 Jan
            • Re: Problems with record visibility and how it is computedJim Starkey28 Jan
              • Another Idea for Transaction Lifetime ControlJim Starkey28 Jan
                • Re: Another Idea for Transaction Lifetime ControlJim Starkey29 Jan
              • RE: Problems with record visibility and how it is computedXuekun Hu4 Feb
          • Re: Problems with record visibility and how it is computedAnn W. Harrison28 Jan
  • Quick question on row countsKeith Murphy24 Jan
    • Re: Quick question on row countsJim Starkey25 Jan
      • Re: Quick question on row countsKeith Murphy25 Jan