From: Sanja Byelkin Date: April 11 2008 8:28am Subject: Plans to make transaction log flush unserialised List-Archive: http://lists.mysql.com/maria/3 Message-Id: <20080411082849.GA9741@desktop.sanja.is.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Hi! Short description of loghandler buffers ======================================= Loghandler has circle of buffers (1-2MB each, 4-5 buffers). Flush procedure get LSN (Log Serial Number, in our case just address in the log) which should be flushed to the disk. Now === First thread which come to flush will lock serialisation mutex check that requested LSN is not already flushed, then go through buffer list and flush it one by one until the LSN become flushed. If it return to the current buffer it will be forced to be closed to write (the last page will be copied to the next buffer to be able continue it). When flussh is domne it will unlock serialisation mutex. Plans ===== First leader ------------ First thread which come to flush (leader of this flush pass) detect buffers which it should flush (beginning and ending / min and max) and set it as goal of current pass. Then it go through buffer list from min to max and flush them one by one skipping already flushed buffers (see 'Flush pass' for details) (Note: A buffer can has overlapping of its first page with the last page of the previous buffer (they appears when we force current buffer to closing). If both are in current plan then thread which flush 'the previous buffer' will not write the last page at all because it is old version of the page) When it reach last buffer it will: - wait of finishing other thread which. - sync file - update information about flushed data - inform threads which are waiting about the pass end. Other threads ------------- Tread which come and see that process of flush is already started - find buffers which should be flushed, if current pass will not satisfy this thread it will check information about next pass and if it need more then already registered there it will update information and set itself as leader of the next pass (like first thread to which come to flush). - then it will take part in flushing buffers (see 'Flush pass' for details). - after which it: - return if all need buffers are flushed (not LSN which could be LSN from future at the moment of flush call which means all current buffers). - become leader of the next pass (if it is still registered as leader) - take part in the next flush pass Flush process status -------------------- - FREE: no threads is flushing - LEADER_DETECT: leader came but goal is not detected yet (other threads just will wait) - FLUSH_PASS: flushing buffer process is in progress - FLUSH_FINISHING: all buffers are at flush we are waiting for the end of the process - FLUSH_END: flush finished leader is syncing and updating statistic - NEW_LEADER: switching to the new leader registered for next pass (if we need new pass) (maybe it will be merged with LEADER_DETECT). Statuses graph -------------- +------------- NEW_LEADER <---------------------------+ | | V | FREE ---> LEADER_DETECT ---> FLUSH_PASS ---> FLUSH_FINISHING --> FLUSH_END ^ | | | +-------------------------------------------------------------------+ Flush pass ---------- - increase number of participant and start loop from min buffer - if buffer flushed skip to the next one if no then flush it - when max buffer of current pass goual processed: decrease participant counter if status is not FLUSH_FINISHING change it. If counter is 0 switch to FLUSH_END. Concerns -------- Switching states and pass through buffers can take a lot of time because different mutexes acquire (but I think it is nothing comparing to IO time). -- __ ___ ___ ____ __ / |/ /_ __/ __/ __ \/ / Mr. Oleksandr Byelkin / /|_/ / // /\ \/ /_/ / /__ MySQL AB, Full-Time Developer /_/ /_/\_, /___/\___\_\___/ Lugansk, Ukraine <___/ www.mysql.com