| List: | Commits | « Previous MessageNext Message » | |
| From: | mcbrown | Date: | November 8 2006 4:14pm |
| Subject: | svn commit - mysqldoc@docsrva: r3882 - trunk/refman-5.1 | ||
| View as plain text | |||
Author: mcbrown Date: 2006-11-08 16:14:42 +0100 (Wed, 08 Nov 2006) New Revision: 3882 Log: Added creating tables section, reformat Modified: trunk/refman-5.1/se-falcon.xml Modified: trunk/refman-5.1/se-falcon.xml =================================================================== --- trunk/refman-5.1/se-falcon.xml 2006-11-08 14:53:37 UTC (rev 3881) +++ trunk/refman-5.1/se-falcon.xml 2006-11-08 15:14:42 UTC (rev 3882) Changed blocks: 3, Lines Added: 771, Lines Deleted: 618; 58838 bytes @@ -11,189 +11,195 @@ ]> <section id="se-falcon"> - <title>Falcon Storage Engine</title> + <title>Falcon Storage Engine</title> + <para> + The Falcon Storage Engine has been designed with modern database + requirements in mind, and particularly for use within high-volume + web serving or other environment that requires high performance, + while still supporting the transactional and logging functionality + required in this environment. + </para> + + <warning> <para> - The Falcon Storage Engine has been designed with modern database - requirements in mind, and particularly for use within high-volume web - serving or other environment that requires high performance, while - still supporting the transactional and logging functionality - required in this environment. + Falcon is currently an Alpha release and should not be used in + production environments. </para> + </warning> - <warning><para>Falcon is currently an Alpha release and should not be used in - production environments.</para></warning> - + <para> + Falcon is currently available only for the Windows and Linux + operating systems running on either 32-bit or 64-bit architectures. + </para> + + <section id="se-falcon-features"> + + <title>Falcon Features</title> + <para> - Falcon is currently available only for the Windows and Linux operating - systems running on either 32-bit or - 64-bit architectures. + Falcon is primarily a memory-based storage engine and has been + specially developed for systems that are able to support larger + memory architectures and multi-threaded or multi-core CPU + environments. Most 64-bit architectures are ideal platforms for + the Falcon engine, where there is a larger available memory space + and 2-,4- or 8-core CPUs available. It can also be deployed within + a standard 32-bit environment. </para> - <section id="se-falcon-features"> + <para> + The Falcon storage engine is designed to work within high-traffic + transactional applications. It supports a number of key features + that make this possible: + </para> - <title>Falcon Features</title> + <itemizedlist> - <para> - Falcon is primarily a memory-based storage engine and has been - specially developed for systems that are able to support larger - memory architectures and multi-threaded or multi-core CPU - environments. Most 64-bit architectures are ideal platforms for - the Falcon engine, where there is a larger available memory - space and 2-,4- or 8-core CPUs available. It can also be - deployed within a standard 32-bit environment. - </para> + <listitem> + <para> + True Multi Version Concurrency Control (MVCC) enables records + and tables to be updated without the overhead associated with + row-level locking mechanisms. The MVCC implementation + virtually eliminates the need to lock tables or rows during + the update process. + </para> + </listitem> - <para> - The Falcon storage engine is designed to work within - high-traffic transactional applications. It supports a number of - key features that make this possible: - </para> + <listitem> + <para> + Flexible locking, including flexible locking levels, smart + deadlock detection and lock timeouts keep data protected and + transactions and operations flowing at full speed. + </para> + </listitem> - <itemizedlist> + <listitem> + <para> + Optimized for modern CPUs and environments to support multiple + threads allowing multiple transactions and fast transaction + handling. + </para> + </listitem> - <listitem> - <para> - True Multi Version Concurrency Control (MVCC) enables - records and tables to be updated without the overhead - associated with row-level locking mechanisms. The MVCC - implementation virtually eliminates the need to lock tables - or rows during the update process. - </para> - </listitem> + <listitem> + <para> + Transaction-safe (fully ACID-compliant) and able to handle + multiple concurrent transactions. + </para> + </listitem> - <listitem> - <para> - Flexible locking, including flexible locking levels, smart deadlock - detection and lock timeouts keep data protected and transactions and - operations flowing at full speed. - </para> - </listitem> - <listitem><para>Optimized for modern CPUs and environments to support - multiple threads allowing multiple transactions and fast transaction - handling. - </para> - </listitem> + <listitem> + <para> + Serial Log provides high performance and recovery capabilities + without sacrificing performance. + </para> + </listitem> - <listitem> - <para> - Transaction-safe (fully ACID-compliant) and able to handle multiple - concurrent transactions. - </para> - </listitem> + <listitem> + <para> + Advanced B-Tree indexes. + </para> + </listitem> - <listitem> - <para> - Serial Log provides high performance and recovery - capabilities without sacrificing performance. - </para> - </listitem> + <listitem> + <para> + Server enforced referential integrity ensures data validity at + all times. + </para> + </listitem> - <listitem> - <para> - Advanced B-Tree indexes. - </para> - </listitem> + <listitem> + <para> + Data compression stores the information on disk in a + compressed format, compressing and decompressing data on the + fly. The result is in smaller and more efficient physical data + sizes. + </para> + </listitem> - <listitem> - <para> - Server enforced referential integrity ensures data validity - at all times. - </para> - </listitem> + <listitem> + <para> + Configurable page sizes enable you to select the page size + according to your data needs. + </para> + </listitem> - <listitem> - <para> - Data compression stores the information on disk in a - compressed format, compressing and decompressing data on the fly. - The result is in smaller and more efficient - physical data sizes. - </para> - </listitem> + <listitem> + <para> + Intelligent disk management automatically manages disk file + size, extensions and space reclamation. + </para> + </listitem> - <listitem> - <para> - Configurable page sizes enable you to select the page size - according to your data needs. - </para> - </listitem> + <listitem> + <para> + Data and index caching provides quick access to data without + the requirement to load index data from disk. + </para> + </listitem> - <listitem> - <para> - Intelligent disk management automatically manages disk file - size, extensions and space reclamation. - </para> - </listitem> + <listitem> + <para> + Implicit savepoints ensure data integrity during transactions. + </para> + </listitem> - <listitem> - <para> - Data and index caching provides quick access to data without - the requirement to load index data from disk. - </para> - </listitem> + </itemizedlist> - <listitem> - <para> - Implicit savepoints ensure data integrity during - transactions. - </para> - </listitem> + </section> - </itemizedlist> + <section id="se-falcon-configuration"> - </section> - - <section id="se-falcon-configuration"> - <title>Configuration Parameters</title> - + <para> - Parameters are configured through the standard - <filename>my.cnf</filename> or <filename>my.ini</filename> file. Parameters can be - configured by specifying the parameter name and the - corresponding value, separated by a space. Memory values can be - specified in bytes, or with a number followed by + Parameters are configured through the standard + <filename>my.cnf</filename> or <filename>my.ini</filename> file. + Parameters can be configured by specifying the parameter name and + the corresponding value, separated by a space. Memory values can + be specified in bytes, or with a number followed by <literal>kb</literal>, <literal>mb</literal> or - <literal>gb</literal>.</para> - - <itemizedlist> - - <listitem> - <para> - <literal>falcon_page_size</literal> (Page Size) controls the - size of the pages used to store information within the - tablespace. Valid sizes are 1, 2, 4, 8, 16 and 32 KB. - </para> - </listitem> - - <listitem> - <para> - <literal>falcon_min_record_memory</literal> (Record Cache Base) - sets the minimum amount of memory that will be allocated for - caching record data. When cache memory is scavenged, the process - will stop when the cache usage reaches this value. The default is - <literal>falcon_max_record_memory</literal>/2 (10MB). - </para> - </listitem> - - <listitem> - <para> - <literal>falcon_max_record_memory</literal> (Record Cache Top) sets - the maximum size of memory that will be allocated for - caching record data. The default is 20MB. - </para> - </listitem> - - <listitem> - <para> - <literal>falcon_page_cache_size</literal> (Page Cache Size) sets - the amount of memory that will be allocated for caching - pages from the tablespace file. The default is 4MB. - </para> - </listitem> - + <literal>gb</literal>. + </para> - <!-- + <itemizedlist> + + <listitem> + <para> + <literal>falcon_page_size</literal> (Page Size) controls the + size of the pages used to store information within the + tablespace. Valid sizes are 1, 2, 4, 8, 16 and 32 KB. + </para> + </listitem> + + <listitem> + <para> + <literal>falcon_min_record_memory</literal> (Record Cache + Base) sets the minimum amount of memory that will be allocated + for caching record data. When cache memory is scavenged, the + process will stop when the cache usage reaches this value. The + default is <literal>falcon_max_record_memory</literal>/2 + (10MB). + </para> + </listitem> + + <listitem> + <para> + <literal>falcon_max_record_memory</literal> (Record Cache Top) + sets the maximum size of memory that will be allocated for + caching record data. The default is 20MB. + </para> + </listitem> + + <listitem> + <para> + <literal>falcon_page_cache_size</literal> (Page Cache Size) + sets the amount of memory that will be allocated for caching + pages from the tablespace file. The default is 4MB. + </para> + </listitem> + +<!-- <remark> [MC] Not currently implemented </remark> @@ -206,21 +212,19 @@ engine for storing information. </para> </listitem>--> - - <listitem> - <para> - <literal>falcon_log_dir</literal> (Log file location) sets - the directory for storing the serial log. The filenames used - by the serial log (two files are create for storing serial - data) are allocated according to the name of the tablespace. - The default location is in the same directory as the - tablespace file. - </para> - </listitem> - - - <!-- + <listitem> + <para> + <literal>falcon_log_dir</literal> (Log file location) sets the + directory for storing the serial log. The filenames used by + the serial log (two files are create for storing serial data) + are allocated according to the name of the tablespace. The + default location is in the same directory as the tablespace + file. + </para> + </listitem> + +<!-- <remark> [MC] Not currently implemented </remark> @@ -232,596 +236,745 @@ log file memory is fixed at 8MB. </para> </listitem>--> - - </itemizedlist> - - <para> - The relationship between the record cache and the page cache is - driven by the information that is cached by each system. Whole - records that are in active use (being read or updated) are - stored within the record cache, however, <literal>BLOB</literal> - data is stored only within the page cache. - </para> - - <para> - The page cache is used to store database metadata, - <literal>BLOB</literal> data and table indexes. - </para> - - <para> - Falcon parameters can be also be set on the command-line to - <command>mysqld</command> using the following command-line - options: - </para> - - <itemizedlist> - - <listitem> - <para> - <literal>--falcon-max-record-memory=#</literal> - </para> - </listitem> - - <listitem> - <para> - <literal>--falcon-min-record-memory=#</literal> - </para> - </listitem> - - <listitem> - <para> - <literal>--falcon-page-cache-size=#</literal> - </para> - </listitem> - - <listitem> - <para> - <literal>--falcon-page-size=#</literal> - </para> - </listitem> - - <listitem> - <para> - <literal>--falcon-log-dir=<replaceable>name</replaceable></literal> - </para> - </listitem> - - </itemizedlist> - - <para> - You can also enable and disable the Falcon storage engine at - startup by supplying these options to <command>mysqld</command>, - providing that the <literal>mysqld</literal> binary includes the - Falcon Storage Engine. - </para> - - <itemizedlist> - - <listitem> - <para> - <literal>--falcon</literal> enables the Falcon storage - engine. - </para> - </listitem> - - <listitem> - <para> - <literal>--skip-falcon</literal> disables the Falcon Storage - Engine. - </para> - </listitem> - - </itemizedlist> - - </section> - - <section id="se-falcon-principles"> - <title>Principles and Terminology</title> + </itemizedlist> - <para> - To get the best out of the Falcon engine you should understand - the following basic principles and terminology. - </para> + <para> + The relationship between the record cache and the page cache is + driven by the information that is cached by each system. Whole + records that are in active use (being read or updated) are stored + within the record cache, however, <literal>BLOB</literal> data is + stored only within the page cache. + </para> - <para>The MySQL Falcon architecture combines advanced techniques with a - simplified structure that results in a high-performance transactional - database that requires little maintenance or troubleshooting by the - database administrator.</para> - - <para></para> - - - <itemizedlist> - <listitem><para><emphasis role="bold">User datafile</emphasis> — stores the Falcon data.</para></listitem> - <listitem><para><emphasis role="bold">Falcon serial log</emphasis> — contains recently committed - data changes, index changes and transactional information. Also - provides data recovery facilities. </para></listitem> - <listitem><para><emphasis role="bold">Page cache</emphasis> — holds database pages being read or written.</para></listitem> - <listitem><para><emphasis role="bold">Record cache</emphasis> — holds copies of active and uncommitted - records.</para></listitem> - <listitem><para><emphasis role="bold">System memory</emphasis> — contains transaction context - information, index accelerators and system metadata.</para></listitem> - <listitem><para><emphasis role="bold">Work Threads</emphasis> — move data from the Falcon Serial Log into - the database page cache and from the page cache to disk. </para></listitem> - </itemizedlist> - - - <section id="se-falcon-principles-tablespace"> + <para> + The page cache is used to store database metadata, + <literal>BLOB</literal> data and table indexes. + </para> - <title>Falcon datafile and data structures</title> + <para> + Falcon parameters can be also be set on the command-line to + <command>mysqld</command> using the following command-line + options: + </para> + <itemizedlist> + + <listitem> <para> - Within Falcon, all data is stored within a single tablespace, - which in turn is stored within a single file within the MySQL - directory structure. A single Falcon database will create three main - files. One file contains the Falcon data and will be stored in a file with - the name of the Falcon database with the extension - <filename>.fts</filename>. For example, Falcon tables defined within - the database <literal>test</literal> will be stored within the file - <filename>test.fts</filename> within the main MySQL data directory.</para> - - <para>Two further files contain the on-disk copy of the Falcon serial - log. These are also created within the realm of the corresponding - database. So with the above example datafile - <filename>test.fts</filename> the log files will be named - <filename>test.fl1</filename> and <filename>test.fl2</filename>.</para> - - <para>Table definitions are, as with with the other MySQL engines, - stored within a <literal>.frm</literal> file within a database - specific directory. For example, the table - <literal>falcontest</literal> within the <literal>test</literal> - database will create the table definition file - <filename>falcontest.frm</filename>.</para> - - <para>A single Falcon database file stores all record - data, indexes, database structure and other information. The - individual information is stored within a series of pages. + <literal>--falcon-max-record-memory=#</literal> </para> - - + </listitem> + + <listitem> <para> - Pages describe the internal storage allocation block within - the Falcon storage engine. Pages are used to store data and - index information. The page size and how the Falcon engine - caches and allocates pages for use when storing information - affect the performance of the engine depending on the records - that are being stored, index complexity + <literal>--falcon-min-record-memory=#</literal> </para> + </listitem> + <listitem> <para> - Pages cached in memory are used to store indexes, blobs and - the structural data for a given tablespace. Active records - (those read or updated are stored within a separate record - cache. + <literal>--falcon-page-cache-size=#</literal> </para> + </listitem> + <listitem> <para> - All transactions on the database are logged and stored within - a separate log file. The logfile is automatically flushed and - the changes written to disk when there is a - <literal>COMMIT</literal> command, when auto-commit is - enabled, or automatically every 30 seconds when transactions - are not being employed. + <literal>--falcon-page-size=#</literal> </para> + </listitem> - </section> + <listitem> + <para> + <literal>--falcon-log-dir=<replaceable>name</replaceable></literal> + </para> + </listitem> - <section id="se-falcon-principles-logging"> + </itemizedlist> - <title>Falcon Serial Log</title> + <para> + You can also enable and disable the Falcon storage engine at + startup by supplying these options to <command>mysqld</command>, + providing that the <literal>mysqld</literal> binary includes the + Falcon Storage Engine. + </para> + <itemizedlist> + + <listitem> <para> - Falcon uses a Serial Log to hold certain types of information - before that data is finally committed to the database. The log - is used to store the following types of information: + <literal>--falcon</literal> enables the Falcon storage engine. </para> + </listitem> - <itemizedlist> + <listitem> + <para> + <literal>--skip-falcon</literal> disables the Falcon Storage + Engine. + </para> + </listitem> - <listitem> - <para> - Data records during the commit phase. - </para> - </listitem> + </itemizedlist> - <listitem> - <para> - Physical database changes required for data recovery after - a crash. - </para> - </listitem> + </section> - <listitem> - <para> - Logical database changes required for resource recovery - after a crash. - </para> - </listitem> + <section id="se-falcon-createdb"> - <listitem> - <para> - Transaction state changes for all active transactions - (active to committed, active to rolled back, active to - limbo). - </para> - </listitem> + <title>Creating the Falcon Tablespace</title> - </itemizedlist> + <para> + Within Falcon, all data is stored within a single tablespace, + which in turn is stored within a single file within the MySQL + directory structure. A single Falcon database will create three + main files. One file contains the Falcon data and will be stored + in a file with the name of the Falcon database with the extension + <filename>.fts</filename>. For example, Falcon tables defined + within the database <literal>test</literal> will be stored within + the file <filename>test.fts</filename> within the main MySQL data + directory. + </para> + <para> + Two further files contain the on-disk copy of the Falcon serial + log. These are also created within the realm of the corresponding + database. So with the above example datafile + <filename>test.fts</filename> the log files will be named + <filename>test.fl1</filename> and <filename>test.fl2</filename>. + </para> + + <para> + Table definitions are, as with with the other MySQL engines, + stored within a <literal>.frm</literal> file within a database + specific directory. For example, the table + <literal>falcontest</literal> within the <literal>test</literal> + database will create the table definition file + <filename>falcontest.frm</filename>. + </para> + + <para> + When creating a table within a MySQL database where the + corresponding Falcon tablespace file does not exist, it will + automatically be created with the Falcon data file and log files. + </para> + + </section> + + <section id="se-falcon-createtable"> + + <title>Creating Tables and Indexes within Falcon</title> + + <para> + Falcon supports all of the standard column datatypes supported by + MySQL. + </para> + + <para> + To create a table that is using the Falcon engine use the + <literal>ENGINE = Falcon</literal> option within the + <literal>CREATE TABLE</literal> statement: + </para> + +<programlisting>CREATE TABLE names (id INT, fname VARCHAR (20), lname VARCHAR (20)) ENGINE=Falcon</programlisting> + + <para> + Indexes can be created using all fothe standard methods; for + example you can explicitly specify an index on a column: + </para> + +<programlisting>CREATE TABLE ids (id int, index (id)) ENGINE=Falcon</programlisting> + + <para> + Generate one as part of a primary key: + </para> + +<programlisting>CREATE TABLE ids (id int),PRIMARY KEY (id) ENGINE=Falcon</programlisting> + + <para> + Or you can create multi-key and multiple indexes: + </para> + +<programlisting>CREATE TABLE t1 (id int NOT NULL,id2 int NOT NULL,id3 int NOT NULL, +name CHAR(30),primary key (id,id2),index index_id3 (id3)) ENGINE=Falcon</programlisting> + + </section> + + <section id="se-falcon-principles"> + + <title>Principles and Terminology</title> + + <para> + To get the best out of the Falcon engine you should understand the + following basic principles and terminology. + </para> + + <para> + The MySQL Falcon architecture combines advanced techniques with a + simplified structure that results in a high-performance + transactional database that requires little maintenance or + troubleshooting by the database administrator. + </para> + + <para></para> + + <itemizedlist> + + <listitem> <para> - All transactions within Falcon are written to the Falcon - Serial Log and then committed to the database, either - automatically when <literal>AUTOCOMMIT</literal> is enabled, - or manually when the <literal>COMMIT</literal> command is - used. + <emphasis role="bold">User datafile</emphasis> — stores + the Falcon data. </para> + </listitem> + <listitem> <para> - Logging information is stored in memory and unwritten changes - to the log are periodically flushed to disk. A background - thread processes the contents of the log, committing the log - changes to the database. The commit process sets the final - status of all records and pages, regardless of any intervening - states; only the final state is actually written to disk. + <emphasis role="bold">Falcon serial log</emphasis> — + contains recently committed data changes, index changes and + transactional information. Also provides data recovery + facilities. </para> + </listitem> + <listitem> <para> - Note, however, that the serial log commit process only updates - the record data through the in-memory page cache. The actual - record data will be written to disk when the checkpoint - process occurs. The exception to this rule are index and blog - entries, which are immediately written to disk as part of the - commit process. + <emphasis role="bold">Page cache</emphasis> — holds + database pages being read or written. </para> + </listitem> + <listitem> <para> - Falcon creates two serial log files. The first log file is - used to store the serial log data until the log reaches a - specified size. Once that size has been reached, logging is - switched to the second serial log file. The commit process - continues to read from the first log file until all - transactions have been written to the database. The first log - file is then released and recreated. + <emphasis role="bold">Record cache</emphasis> — holds + copies of active and uncommitted records. </para> + </listitem> + <listitem> <para> - Log entries in the second file are then processed until all - transactions in the log have been completed. That file is then - released and recreated, ready to be pressed into use as soon - as the first log file is full or becomes locked for commits. + <emphasis role="bold">System memory</emphasis> — + contains transaction context information, index accelerators + and system metadata. </para> + </listitem> - <section id="se-falcon-principles-logging-rollback"> + <listitem> + <para> + <emphasis role="bold">Work Threads</emphasis> — move + data from the Falcon Serial Log into the database page cache + and from the page cache to disk. + </para> + </listitem> - <title>Rollback Process</title> + </itemizedlist> - <para> - Transaction rollbacks are handled by the thread for that - transaction. The rollback process performs the following - actions: - </para> + <section id="se-falcon-principles-tablespace"> - <itemizedlist> + <title>Falcon datafile and data structures</title> - <listitem> - <para> - Backing out index updates. - </para> - </listitem> + <para> + A single Falcon database file stores all record data, indexes, + database structure and other information. The individual + information is stored within a series of pages. + </para> - <listitem> - <para> - Backing out any blob data created by the transaction. - </para> - </listitem> + <para> + Pages describe the internal storage allocation block within the + Falcon storage engine. Pages are used to store data and index + information. The page size and how the Falcon engine caches and + allocates pages for use when storing information affect the + performance of the engine depending on the records that are + being stored, index complexity + </para> - <listitem> - <para> - Releasing allocated record slots. - </para> - </listitem> + <para> + Pages cached in memory are used to store indexes, blobs and the + structural data for a given tablespace. Active records (those + read or updated are stored within a separate record cache. + </para> - <listitem> - <para> - Backing out record versions created in memory. - </para> - </listitem> + <para> + All transactions on the database are logged and stored within a + separate log file. The logfile is automatically flushed and the + changes written to disk when there is a + <literal>COMMIT</literal> command, when auto-commit is enabled, + or automatically every 30 seconds when transactions are not + being employed. + </para> - </itemizedlist> + </section> - </section> + <section id="se-falcon-principles-logging"> - <section id="se-falcon-principles-logging-groupcommit"> + <title>Falcon Serial Log</title> - <title>Group Commits</title> + <para> + Falcon uses a Serial Log to hold certain types of information + before that data is finally committed to the database. The log + is used to store the following types of information: + </para> + <itemizedlist> + + <listitem> <para> - For performance, Falcon uses a group commit system that - ensures that all pending updates to the serial log are - written to disk at the same time. Falcon can have multiple - active transactions, but only one transactions writes all - the pending changes into the serial log on disk, reducing - the number of disk writes and improving the overall - performance of the serial log. + Data records during the commit phase. </para> + </listitem> + <listitem> <para> - For example: + Physical database changes required for data recovery after a + crash. </para> + </listitem> - <orderedlist> + <listitem> + <para> + Logical database changes required for resource recovery + after a crash. + </para> + </listitem> - <listitem> - <para> - Transaction 1 commits, creates all the necessary log - entries, and starts to write the log to disk. - </para> - </listitem> + <listitem> + <para> + Transaction state changes for all active transactions + (active to committed, active to rolled back, active to + limbo). + </para> + </listitem> - <listitem> - <para> - Whilte Transaction 1 commits are being written, - Transacations 2 and 3 write their log entries into the - serial log. - </para> - </listitem> + </itemizedlist> - <listitem> - <para> - Once Transaction 1 has finished the physical write, - either transaction 2 or 3 (but not both) will write out - the unwritten portion of the in-memory log to disk. - Because both transacations have occurred since the last - disk-write of the serial log, the informaiton for both - is written to the disk at the same time. - </para> - </listitem> + <para> + All transactions within Falcon are written to the Falcon Serial + Log and then committed to the database, either automatically + when <literal>AUTOCOMMIT</literal> is enabled, or manually when + the <literal>COMMIT</literal> command is used. + </para> - <listitem> - <para> - While transactions 2 and 3 are writing, transactions 4, - 5 and 6 are being written to the in-memory log. When the - write for 2 and 3 completes, the entries for 4, 5 and 6 - are written. - </para> - </listitem> + <para> + Logging information is stored in memory and unwritten changes to + the log are periodically flushed to disk. A background thread + processes the contents of the log, committing the log changes to + the database. The commit process sets the final status of all + records and pages, regardless of any intervening states; only + the final state is actually written to disk. + </para> - </orderedlist> + <para> + Note, however, that the serial log commit process only updates + the record data through the in-memory page cache. The actual + record data will be written to disk when the checkpoint process + occurs. The exception to this rule are index and blog entries, + which are immediately written to disk as part of the commit + process. + </para> - <para> - The result of the above process is that there are only three - physical writes to disk, even though there are six - transactions in the sequence: - </para> + <para> + Falcon creates two serial log files. The first log file is used + to store the serial log data until the log reaches a specified + size. Once that size has been reached, logging is switched to + the second serial log file. The commit process continues to read + from the first log file until all transactions have been written + to the database. The first log file is then released and + recreated. + </para> - <itemizedlist> + <para> + Log entries in the second file are then processed until all + transactions in the log have been completed. That file is then + released and recreated, ready to be pressed into use as soon as + the first log file is full or becomes locked for commits. + </para> - <listitem> - <para> - Transaction 1 - </para> - </listitem> + <section id="se-falcon-principles-logging-rollback"> - <listitem> - <para> - Transactions 2 and 3 - </para> - </listitem> + <title>Rollback Process</title> - <listitem> - <para> - Transactions 4, 5 and 6 - </para> - </listitem> + <para> + Transaction rollbacks are handled by the thread for that + transaction. The rollback process performs the following + actions: + </para> - </itemizedlist> + <itemizedlist> - <para> - The process continues, with just one transaction writing all - the in-memory serial log entries to disk since the last - write. The entire system ensures that the in-memory and the - disk log are kept in synchronization, with the lowest amount - of physical disk writes - </para> + <listitem> + <para> + Backing out index updates. + </para> + </listitem> - <para> - The above process works in tandem with the use of the two - serial log files to ensure that the in-memory information - and on disk information are kiept up to date. - </para> + <listitem> + <para> + Backing out any blob data created by the transaction. + </para> + </listitem> - </section> + <listitem> + <para> + Releasing allocated record slots. + </para> + </listitem> + <listitem> + <para> + Backing out record versions created in memory. + </para> + </listitem> + + </itemizedlist> + </section> - <section id="se-falcon-principles-recovery"> + <section id="se-falcon-principles-logging-groupcommit"> - <title>Falcon Crash Recovery</title> + <title>Group Commits</title> <para> - The Falcon Serial Log is used automatically during startup to - recover transactions and update the database. When - transactions and changes are written to the Serial Log the log - includes entries that record changes to all areas of the - database, including the indexes, changes to - <literal>BLOB</literal> data, and any structural changes to - the database. + For performance, Falcon uses a group commit system that + ensures that all pending updates to the serial log are written + to disk at the same time. Falcon can have multiple active + transactions, but only one transactions writes all the pending + changes into the serial log on disk, reducing the number of + disk writes and improving the overall performance of the + serial log. </para> <para> - During crash recovery, Falcon examines the serial log and - identifies the first entry that has not been committed to the - database. The recovery process wriutes all unwritten data, - changes index and blob data, releasing any necessary record - slots (from deleted records) and committing any structural - changes. + For example: </para> - </section> + <orderedlist> - <section id="se-falcon-principles-caches"> + <listitem> + <para> + Transaction 1 commits, creates all the necessary log + entries, and starts to write the log to disk. + </para> + </listitem> - <title>Falcon Memory Caches</title> - - <para>Falcon was designed to perform best on systems with generous amounts of memory. The memory - caches utilized by Falcon are similar in some respects with other RDBMS’s and MySQL engines; - however, the cache structures offer a number of improvements over traditional memory caching - strategies. The mechanisms used by Falcon with respect to memory caching include:</para> - - <itemizedlist> <listitem><para><emphasis role="bold">Log Cache</emphasis> — log information is kept in memory and flushed to the Falcon Log when - transactions commit. Falcon keeps eight windows into the log file for reading and writing, and - each window is 1MB. </para> </listitem> - <listitem> <para><emphasis role="bold">System and Index Cache</emphasis> — data needed by Falcon (table and field definitions, transaction - state, etc.) are also maintained in memory for quick reference. In addition, local index - accelerators represent index segments created by a running transaction are also stored in the - system memory. When a transaction changes indexed fields, it builds an index accelerator - section in system memory, representing its changes. On commit, all index changes for the - transaction are written to the serial log in sorted order and later merged with the permanent - index by the worker thread. </para> </listitem> - <listitem> <para><emphasis role="bold">Page Cache</emphasis> — database pages read from disk for a particular database. The page cache - size is controlled by the <literal>falcon_page_cache_size</literal> parameter, the default of which is 4MB, - and is set in the my.cnf file. Although record and index changes go to the serial log before - being written to database pages, blob data is written directly into the page cache. This avoids - logging large data items that are rarely referenced or changed by the transaction that creates - them.</para> </listitem> - <listitem> <para><emphasis role="bold">Record Cache</emphasis> — the record cache is a memory region devoted to holding rows that have - been requested by end-user queries for a particular database or created by active - transactions. Note that this cache differs from traditional data caches in that only specific - rows needed by applications reside in the cache as opposed to entire data pages (which may - contain only subsets of needed information). The record cache can hold several versions of - records that have been modified or deleted. This technique guarantees that active data - needed to satisfy user requests is in memory, shortens row access time, and reduces cache - bloat by not including un-requested information. The record cache also assists in supporting - the multi-version concurrency control (MVCC) mechanisms of the Falcon engine. The record - cache is controlled by two parameters. The <literal>falcon_min_record_memory</literal> parameter - (default 10MB) determines the minimum amount of RAM supplied to the record cache, and - the <literal>falcon_max_record_memory</literal> (default 20MB) limits the total amount of memory - available to the cache.</para> </listitem> - - <listitem><para>Because of the support the record cache supplies to transactions, a scavenge thread is used to - ensure only “hot” data resides in the cache. When the <literal>falcon_max_record_memory</literal> limit is - reached, Falcon surveys the demographics of the generational data in the cache, and removes the - oldest generations. This process is more complicated than the standard LRU algorithm used by many - database systems, but it is more efficient and faster.</para> </listitem></itemizedlist> - - </section> - - <section id="se-falcon-principles-threads"> - <title>Falcon Threads</title> - - <para>Falcon uses two worker threads to process information within the - Falcon structures. One thread, the "gopher" thread, is devoted to - moving committed data changes from the Falcon log to data pages and to - merge index changes with permanent index data. The second thread handles - the periodic flushing of the page cache and scavenges space allocated - within the record cache. </para> - - </section> - - <section id="se-falcon-principles-compression"> + <listitem> + <para> + Whilte Transaction 1 commits are being written, + Transacations 2 and 3 write their log entries into the + serial log. + </para> + </listitem> - <title>Data Compression</title> + <listitem> + <para> + Once Transaction 1 has finished the physical write, either + transaction 2 or 3 (but not both) will write out the + unwritten portion of the in-memory log to disk. Because + both transacations have occurred since the last disk-write + of the serial log, the informaiton for both is written to + the disk at the same time. + </para> + </listitem> + <listitem> + <para> + While transactions 2 and 3 are writing, transactions 4, 5 + and 6 are being written to the in-memory log. When the + write for 2 and 3 completes, the entries for 4, 5 and 6 + are written. + </para> + </listitem> + + </orderedlist> + <para> - Data stored in the Falcon tablespace is compressed on disk, - but is stored in an uncompressed format in memory. Compression - occurs automatically when data is committed to disk. + The result of the above process is that there are only three + physical writes to disk, even though there are six + transactions in the sequence: </para> - </section> + <itemizedlist> - <section id="se-falcon-principles-recordslot"> + <listitem> + <para> + Transaction 1 + </para> + </listitem> - <title>Record Slot</title> + <listitem> + <para> + Transactions 2 and 3 + </para> + </listitem> + <listitem> + <para> + Transactions 4, 5 and 6 + </para> + </listitem> + + </itemizedlist> + <para> - A record slot is an internal record identifier that is used to - find records in memory and on disk. It is essentially a - pointer to the pages that contain the data for a particular - record. A new record slot is created for each record for the - duration of that record's existence. The record slot is only - relinquished when the record is erased from the database. + The process continues, with just one transaction writing all + the in-memory serial log entries to disk since the last write. + The entire system ensures that the in-memory and the disk log + are kept in synchronization, with the lowest amount of + physical disk writes </para> + <para> + The above process works in tandem with the use of the two + serial log files to ensure that the in-memory information and + on disk information are kiept up to date. + </para> + </section> </section> - - <section id="se-falcon-limits"> - <title>Limits</title> + <section id="se-falcon-principles-recovery"> + <title>Falcon Crash Recovery</title> + <para> - There are a number of limits in the current version of Falcon: + The Falcon Serial Log is used automatically during startup to + recover transactions and update the database. When transactions + and changes are written to the Serial Log the log includes + entries that record changes to all areas of the database, + including the indexes, changes to <literal>BLOB</literal> data, + and any structural changes to the database. </para> + <para> + During crash recovery, Falcon examines the serial log and + identifies the first entry that has not been committed to the + database. The recovery process wriutes all unwritten data, + changes index and blob data, releasing any necessary record + slots (from deleted records) and committing any structural + changes. + </para> + + </section> + + <section id="se-falcon-principles-caches"> + + <title>Falcon Memory Caches</title> + + <para> + Falcon was designed to perform best on systems with generous + amounts of memory. The memory caches utilized by Falcon are + similar in some respects with other RDBMS’s and MySQL engines; + however, the cache structures offer a number of improvements + over traditional memory caching strategies. The mechanisms used + by Falcon with respect to memory caching include: + </para> + <itemizedlist> <listitem> <para> - There is a limit of 2<superscript>32</superscript> (4.29 - billion) records for a single table. By using multiple - tables within the same tablespace you can have more than - this number of records. + <emphasis role="bold">Log Cache</emphasis> — log + information is kept in memory and flushed to the Falcon Log + when transactions commit. Falcon keeps eight windows into + the log file for reading and writing, and each window is + 1MB. </para> </listitem> <listitem> <para> - Pages can be a maximum of 32768 bytes in size. + <emphasis role="bold">System and Index Cache</emphasis> + — data needed by Falcon (table and field definitions, + transaction state, etc.) are also maintained in memory for + quick reference. In addition, local index accelerators + represent index segments created by a running transaction + are also stored in the system memory. When a transaction + changes indexed fields, it builds an index accelerator + section in system memory, representing its changes. On + commit, all index changes for the transaction are written to + the serial log in sorted order and later merged with the + permanent index by the worker thread. </para> </listitem> <listitem> <para> - Each tablespace has a limit of - 2<superscript>32</superscript> pages within a single - tablespace. Through a combination of the page size and the - maximum number of pages in a tablespace, there is a limit of - 140,737,488,355,328 bytes (128 TB) to a single tablespace. + <emphasis role="bold">Page Cache</emphasis> — database + pages read from disk for a particular database. The page + cache size is controlled by the + <literal>falcon_page_cache_size</literal> parameter, the + default of which is 4MB, and is set in the my.cnf file. + Although record and index changes go to the serial log + before being written to database pages, blob data is written + directly into the page cache. This avoids logging large data + items that are rarely referenced or changed by the + transaction that creates them. </para> </listitem> - </itemizedlist> - - <para> - Although the maximum available storage within a tablespace is - 128TB, the true number of records and quantity of data that you - can store is dependent on a number of factors: - </para> - - <itemizedlist> - <listitem> <para> - Record storage requirements + <emphasis role="bold">Record Cache</emphasis> — the + record cache is a memory region devoted to holding rows that + have been requested by end-user queries for a particular + database or created by active transactions. Note that this + cache differs from traditional data caches in that only + specific rows needed by applications reside in the cache as + opposed to entire data pages (which may contain only subsets + of needed information). The record cache can hold several + versions of records that have been modified or deleted. This + technique guarantees that active data needed to satisfy user + requests is in memory, shortens row access time, and reduces + cache bloat by not including un-requested information. The + record cache also assists in supporting the multi-version + concurrency control (MVCC) mechanisms of the Falcon engine. + The record cache is controlled by two parameters. The + <literal>falcon_min_record_memory</literal> parameter + (default 10MB) determines the minimum amount of RAM supplied + to the record cache, and the + <literal>falcon_max_record_memory</literal> (default 20MB) + limits the total amount of memory available to the cache. </para> </listitem> <listitem> <para> - Index storage requirements + Because of the support the record cache supplies to + transactions, a scavenge thread is used to ensure only + “hot” data resides in the cache. When the + <literal>falcon_max_record_memory</literal> limit is + reached, Falcon surveys the demographics of the generational + data in the cache, and removes the oldest generations. This + process is more complicated than the standard LRU algorithm + used by many database systems, but it is more efficient and + faster. </para> </listitem> - <listitem> - <para> - Compression ratio of stored data - </para> - </listitem> - </itemizedlist> + </section> + + <section id="se-falcon-principles-threads"> + + <title>Falcon Threads</title> + <para> - Because of the complex relationship between the storage, - indexing and compression facilities it is impossible to predict - or calculate the disk storage space required for a specific - dataset. + Falcon uses two worker threads to process information within the + Falcon structures. One thread, the "gopher" thread, is devoted + to moving committed data changes from the Falcon log to data + pages and to merge index changes with permanent index data. The + second thread handles the periodic flushing of the page cache + and scavenges space allocated within the record cache. </para> </section> + <section id="se-falcon-principles-compression"> + + <title>Data Compression</title> + + <para> + Data stored in the Falcon tablespace is compressed on disk, but + is stored in an uncompressed format in memory. Compression + occurs automatically when data is committed to disk. + </para> + + </section> + + <section id="se-falcon-principles-recordslot"> + + <title>Record Slot</title> + + <para> + A record slot is an internal record identifier that is used to + find records in memory and on disk. It is essentially a pointer + to the pages that contain the data for a particular record. A + new record slot is created for each record for the duration of + that record's existence. The record slot is only relinquished + when the record is erased from the database. + </para> + + </section> + + </section> + + <section id="se-falcon-limits"> + + <title>Limits</title> + + <para> + There are a number of limits in the current version of Falcon: + </para> + + <itemizedlist> + + <listitem> + <para> + There is a limit of 2<superscript>32</superscript> (4.29 + billion) rows for a single table. By using multiple tables + within the same tablespace you can have more than this number + of records. + </para> + </listitem> + + <listitem> + <para> + Pages can be a maximum of 32768 bytes in size. + </para> + </listitem> + + <listitem> + <para> + Falcon tables can support up to 32,0000 columns. + </para> + </listitem> + + <listitem> + <para> + Each tablespace has a limit of 2<superscript>32</superscript> + pages within a single tablespace. Through a combination of the + page size and the maximum number of pages in a tablespace, + there is a limit of 140,737,488,355,328 bytes (128 TB) to a + single tablespace. + </para> + </listitem> + + </itemizedlist> + + <para> + Although the maximum available storage within a tablespace is + 128TB, the true number of records and quantity of data that you + can store is dependent on a number of factors: + </para> + + <itemizedlist> + + <listitem> + <para> + Record storage requirements + </para> + </listitem> + + <listitem> + <para> + Index storage requirements + </para> + </listitem> + + <listitem> + <para> + Compression ratio of stored data + </para> + </listitem> + + </itemizedlist> + + <para> + Because of the complex relationship between the storage, indexing + and compression facilities it is impossible to predict or + calculate the disk storage space required for a specific dataset. + </para> + + </section> + <!--> <section id="se-falcon-roadmap"> <title>Falcon Roadmap</title> <para>Falcon will have...</para>
| Thread | ||
|---|---|---|
| • svn commit - mysqldoc@docsrva: r3882 - trunk/refman-5.1 | mcbrown | 8 Nov |
© 1995-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
Page generated in 1.085 sec. using MySQL 5.1.14-beta-log