List:Maria Storage Engine« Previous MessageNext Message »
From:Sanja Byelkin Date:April 11 2008 8:28am
Subject:Plans to make transaction log flush unserialised
View as plain text  

Short description of loghandler buffers

Loghandler has circle of buffers (1-2MB each, 4-5 buffers).

Flush procedure get LSN (Log Serial Number, in our case just address in
the log) which should be flushed to the disk. 


First thread which come to flush will lock serialisation mutex check
that requested LSN is not already flushed, then go through buffer list
and flush it one by one until the LSN become flushed. If it return to
the current buffer it will be forced to be closed to write (the last
page will be copied to the next buffer to be able continue it). 
When flussh is domne it will unlock serialisation mutex.


First leader

First thread which come to flush (leader of this flush pass) detect
buffers which it should flush (beginning and ending / min and max) and
set it as goal of current pass.

Then it go through buffer list from min to max  and flush them one by
one skipping already flushed buffers (see 'Flush pass' for details)
 (Note: A buffer can has overlapping of its first page with the last
 page of the previous buffer (they appears when we force current buffer
 to closing). If both are in current plan then thread which flush 'the
 previous buffer' will not write the last page at all because it is old
 version of the page)

When it reach last buffer it will:
 - wait of finishing other thread which.
 - sync file
 - update information about flushed data
 - inform threads which are waiting about the pass end.

Other threads

Tread which come and see that process of flush is already started
- find buffers which should be flushed, if current pass will not satisfy
  this thread it will check information about next pass and if it need
  more then already registered there it  will update information and set
  itself as leader of the next pass (like first thread to which come to
- then it will take part in flushing buffers (see 'Flush pass' for
- after which it:
  - return if all need buffers are flushed (not LSN which could be LSN
    from future at the moment of flush call which means all current
  - become leader of the next pass (if it is still registered as leader)
  - take part in the next flush pass

Flush process status

 - FREE: no threads is flushing
 - LEADER_DETECT: leader came but goal is not detected yet (other
   threads just will wait)
 - FLUSH_PASS: flushing buffer process is in progress
 - FLUSH_FINISHING: all buffers are at flush we are waiting for the end
   of the process
 - FLUSH_END: flush finished leader is syncing and updating statistic
 - NEW_LEADER: switching to the new leader registered for next pass
   (if we need new pass) (maybe it will be merged with LEADER_DETECT).

Statuses graph

               +------------- NEW_LEADER <---------------------------+
               |                                                     |
               V                                                     |
 ^                                                                   |
 |                                                                   |

Flush pass

- increase number of participant and start loop from min buffer
  - if buffer flushed skip to the next one if no then flush it
- when max buffer of current pass goual processed:  decrease participant counter if status
is not
  FLUSH_FINISHING change it. If counter is 0 switch to FLUSH_END.


Switching states and pass through buffers can take a lot of time because
different mutexes acquire (but I think it is nothing comparing to IO time).

   __  ___     ___ ____  __
  /  |/  /_ __/ __/ __ \/ /    Mr. Oleksandr Byelkin <sanja@stripped>
 / /|_/ / // /\ \/ /_/ / /__   MySQL AB, Full-Time Developer
/_/  /_/\_, /___/\___\_\___/   Lugansk, Ukraine
Plans to make transaction log flush unserialisedSanja Byelkin11 Apr
  • Re: Plans to make transaction log flush unserialisedOleksandr \"Sanja\" Byelkin17 Apr
  • re: Plans to make transaction log flush unserialisedMichael Widenius7 May
    • Re: Plans to make transaction log flush unserialisedSanja Byelkin8 May
      • Re: Plans to make transaction log flush unserialisedMichael Widenius8 May
        • Re: Plans to make transaction log flush unserialisedSanja Byelkin8 May