List:Falcon Storage Engine« Previous MessageNext Message »
From:Kevin Lewis Date:October 30 2008 12:55am
Subject:Re: Unthawable
View as plain text  
Phillip, every database system I know suffers from this problem that you 
cannot get rid of old parts of the serial log until the old parts do not 
contain anything from a committed transaction that is not yet on disk in 
page form.  Pervasive.SQL uses small 500Kb files that keep adding up and 
filling up the disk.  After the checkpoint, the old ones can be deleted.

We have the same problem, except we have only two files at a time.  It 
is always up to the DBA or application designer to try to optimize the 
database and queries in a way that transactions do not stay open for 
long enough that the serial log will fill up the disk.

I think Ann's comment is referring to the likelihood that chilled 
records will need to be deleted.  Yesterday, I repeated and debugged (a 
little) the current issue that Chris is seeing with unthawable records. 
currently tracked by Bug#39696)  We only see it using your Gentest and 
falcon-record-chill-threshold=4Kb or less.  Sometimes, during this test 
the same record version will chill and thaw several times.  It is a 
wonderful stress test.  Even though it is not likely to see in the real 
world, given enough time, what can happen, WILL HAPPEN!

Kevin

Philip Stoev wrote:
> 
>>> Philip's cases are valuable tests, but not typical usage...  With
>>> reasonable settings, chills will happen only during loads and mass
>>> inserts.  The chances of modifying or deleting a record that's been
>>> chilled are small, and the chances of having a multi-level modify /
>>> chill / delete cycle are lower still.
> 
> Please find below an email from Kevin on another thread that describes a 
> potential chill/thaw/backlogging scenario that is not a load or a mass 
> insert -- attempting to generate a report by making multiple passes 
> through the entire data while writing intermediate values in a summary 
> table. If you decide to mark the start of your execution by inserting a 
> timestamp in some log table, your transaction has become an update one 
> and thus the serial log will grow and the memory will fill up with old 
> record versions.
> 
> Philip Stoev
> 
> ----- Original Message ----- From: "Kevin Lewis" <Kevin.Lewis@stripped>
> To: "Ingo Strüwing" <Ingo.Struewing@stripped>
> Cc: "Øystein Grøvlen" <Oystein.Grovlen@stripped>; "'dev-backup'" 
> <dev-backup@stripped>; <falcon-private@stripped>
> Sent: Wednesday, October 29, 2008 9:28 PM
> Subject: Re: Restore makes ongoing transactions see empty tables
> 
>> A single transaction may make a couple passes through a table summing 
>> up certain columns for certain criteria.  We do not want that data to 
>> change in the middle of that transaction, for example, between pass 
>> one and pass two.  Currently, a table will be truncated and then the 
>> restored data will get replaced within a new transaction.  The 
>> truncate will be seen immediately by the existing transaction, but the 
>> restored data, being transactional, will not be seen. 
> 
> 
Thread
UnthawableKevin Lewis25 Oct
  • Re: UnthawableJim Starkey25 Oct
  • Re: UnthawablePhilip Stoev29 Oct
    • Re: UnthawableKevin Lewis30 Oct
      • Re: UnthawableLars-Erik Bjørk30 Oct
    • Re: UnthawableAnn W. Harrison30 Oct
  • Re: UnthawablePhilip Stoev30 Oct
    • Re: UnthawableKevin Lewis30 Oct
      • Re: UnthawableHakan Kuecuekyilmaz30 Oct
      • Re: UnthawableJames Day31 Oct
Re: UnthawableKevin Lewis27 Oct
  • Re: UnthawableAnn W. Harrison27 Oct
  • Re: UnthawableJim Starkey27 Oct