Author: jstephens
Date: 2008-05-21 17:12:31 +0200 (Wed, 21 May 2008)
New Revision: 10799
Log:
Rewrite of 5.1/NDB 6.x Cluster Features (mysql-cluster-roadmap)
- Changed titles (to help avoid confusion with changelogs)
- Cut items of no major importance
- Restructured with <formalpara>s (for inline titles)
- Expanded descriptions
- Added links for most items to more info in main part of
Cluster chapter
- Added new item descriptions:
- Distribution Awareness
- Realtime Extensions
- Epoll Support on Linux
Updated dependencies
Modified:
trunk/it/refman-5.1/Makefile.depends
trunk/it/refman-5.1/mysql-cluster-roadmap.xml
trunk/pt/refman-5.1/Makefile.depends
trunk/pt/refman-5.1/mysql-cluster-roadmap.xml
trunk/refman-5.1-maria/Makefile.depends
trunk/refman-5.1/Makefile.depends
trunk/refman-5.1/mysql-cluster-roadmap.xml
Modified: trunk/it/refman-5.1/Makefile.depends
===================================================================
--- trunk/it/refman-5.1/Makefile.depends 2008-05-21 12:39:50 UTC (rev 10798)
+++ trunk/it/refman-5.1/Makefile.depends 2008-05-21 15:12:31 UTC (rev 10799)
Changed blocks: 3, Lines Added: 9, Lines Deleted: 0; 1882 bytes
@@ -1754,6 +1754,7 @@
../../ndbapi/metadata/class-ndboperation.idmap \
../../ndbapi/metadata/class-ndbtransaction.idmap \
../../ndbapi/metadata/class-table.idmap \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
../../ndbapi/metadata/mgm-api.idmap \
../../ndbapi/metadata/ndb-errors.idmap \
../../ndbapi/metadata/ndb-internals.idmap \
@@ -2194,9 +2195,16 @@
mysql_cluster_roadmap_IMAGES =
mysql_cluster_roadmap_SOURCES = mysql-cluster-roadmap.xml
$(mysql_cluster_roadmap_INCLUDES)
mysql_cluster_roadmap_IDMAPS = \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
+ ../../ndbapi/metadata/ndb-internals.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-backup.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-configuration.idmap \
../../refman-5.1/metadata/mysql-cluster-disk-data.idmap \
../../refman-5.1/metadata/mysql-cluster-limitations.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-management.idmap \
../../refman-5.1/metadata/mysql-cluster-news-core.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-optvar-core.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-process-management.idmap \
../../refman-5.1/metadata/mysql-cluster-replication.idmap \
../../refman-5.1/metadata/mysql-cluster-roadmap.idmap \
../../refman-5.1/metadata/partitioning.idmap \
@@ -2358,6 +2366,7 @@
../../ndbapi/metadata/class-dictionary.idmap \
../../ndbapi/metadata/class-ndb-cluster-connection.idmap \
../../ndbapi/metadata/class-ndb.idmap \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
../../ndbapi/metadata/mgm-api.idmap \
../../ndbapi/metadata/ndb-errors.idmap \
../../ndbapi/metadata/ndb-internals.idmap \
Modified: trunk/it/refman-5.1/mysql-cluster-roadmap.xml
===================================================================
--- trunk/it/refman-5.1/mysql-cluster-roadmap.xml 2008-05-21 12:39:50 UTC (rev 10798)
+++ trunk/it/refman-5.1/mysql-cluster-roadmap.xml 2008-05-21 15:12:31 UTC (rev 10799)
Changed blocks: 12, Lines Added: 866, Lines Deleted: 551; 66769 bytes
@@ -19,11 +19,6 @@
<primary>New features in MySQL Cluster</primary>
</indexterm>
- <remark role="todo">
- [js] Update to get rid of items already implemented that were listed
- as "future" items here. (Waiting on feedback from Jeb, et al.)
- </remark>
-
<remark role="note">
[js] Altered previously version-specific section ID, title.
</remark>
@@ -31,35 +26,143 @@
<remark>
Author: Jon Stephens. Based on Mikael Ronström's "Cluster News"
announcement of 2005-04-06 and other info from Mikael Ronström,
- Jonas Oreland, et al.
+ Jonas Oreland, et al. Updated with information from NDB developers
+ and support staff; May 2008 reorganisation inspired and assisted by
+ Johan Andersson.
</remark>
- <remark role="todo">
- [pd] I see nothing for next-series. [js] Commented out reference for
- now.
- </remark>
-
<para>
In this section, we discuss changes in the implementation of MySQL
Cluster in MySQL ¤t-series; and &mccge-series; as compared to
MySQL &previous-series;.
+ </para>
-<!--
- We also discuss our roadmap for further improvements to MySQL Cluster
- as currently planned for MySQL &next-series;.
+<!--
+ <para>
+
+ <remark role="TODO">
+ [js] Uncomment and expand this para when we have some likely future
+ improvements to describe, or possibly add new section for these.
+ </remark>
+ We also discuss our roadmap for further improvements to MySQL Cluster
+ as currently planned for MySQL &next-series;.
+ </para>
-->
- </para>
<para>
There are a number of significant changes in the implementation of
- the <literal>NDBCLUSTER</literal> storage engine in MySQL 5.1 as
- compared to that in MySQL 5.0. For an overview of these changes, see
- <xref linkend="mysql-cluster-changes-5-1"/>
+ the <literal>NDBCLUSTER</literal> storage engine in mainline MySQL
+ 5.1 releases up to and including MySQL 5.1.23 as compared to that in
+ MySQL 5.0; &mccge-series; makes further changes and improvements in
+ MySQL Cluster in addition to these. The changes and features most
+ likely to be of interest are shown in the following table:
+
+ <remark role="TODO">
+ [js] Add IDs to items in subsections and make entries in this
+ table links to these...
+ </remark>
+
+ <informaltable>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link linkend="mysql-cluster-changes-5-1">MySQL 5.1
(through
+ 5.1.23)</link></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>MySQL Cluster Replication</entry>
+ </row>
+ <row>
+ <entry>Disk Data storage</entry>
+ </row>
+ <row>
+ <entry>Variable-size columns</entry>
+ </row>
+ <row>
+ <entry>User-defined partitioning</entry>
+ </row>
+ <row>
+ <entry>Autodiscovery of table schema changes</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link
linkend="mysql-cluster-changes-5-1-ndb-6-1">&mccge-series;
+ 6.1</link></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Greater number of cluster nodes</entry>
+ </row>
+ <row>
+ <entry>Disabling of arbitration</entry>
+ </row>
+ <row>
+ <entry>Additional <literal>DUMP</literal>
commands</entry>
+ </row>
+ <row>
+ <entry>Faster Disk Data backups</entry>
+ </row>
+ <row>
+ <entry>Batched slave updates</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link
linkend="mysql-cluster-changes-5-1-ndb-6-2">&mccge-series;
+ 6.2</link></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>y</entry>
+ </row>
+ <row>
+ <entry>y</entry>
+ </row>
+ <row>
+ <entry>x</entry>
+ <entry>y</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link
linkend="mysql-cluster-changes-5-1-ndb-6-3">&mccge-series;
+ 6.3</link></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>y</entry>
+ </row>
+ <row>
+ <entry>y</entry>
+ </row>
+ <row>
+ <entry>x</entry>
+ <entry>y</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
</para>
<section id="mysql-cluster-changes-5-1">
- <title>MySQL Cluster Changes in MySQL 5.1</title>
+ <title>Features Added in MySQL 5.1 Cluster</title>
<indexterm>
<primary>MySQL Cluster</primary>
@@ -68,7 +171,10 @@
<para>
A number of new features for MySQL Cluster have been implemented
- in MySQL 5.1:
+ in MySQL 5.1 through MySQL 5.1.23, when support for MySQL Cluster
+ was moved to &mccge-series;. All of the features in the following
+ list are also available in all &mccge-series; (6.1 and later)
+ releases.
</para>
<itemizedlist>
@@ -79,11 +185,21 @@
<title>Integration of MySQL Cluster into MySQL Replication</title>
<para>
- This makes it possible to update from any MySQL Server in
- the cluster and still have the MySQL Replication handled by
- one of the MySQL Servers in the cluster, with the state of
- the slave side remaining consistent with the cluster acting
- as the master.
+ MySQL Cluster Replication makes it possible to replicate
+ from one MySQL Cluster to another. Updates on any SQL node
+ (MySQL server) in the cluster acting as the master are
+ replicated to the slave cluster; the state of the slave side
+ remains consistent with the cluster acting as the master.
+ This is sometimes referred to as <firstterm>asynchronous
+ replication</firstterm> between clusters, providing
+ <firstterm>geographic redundancy</firstterm>. It is also
+ possible to replicate from a MySQL Cluster acting as the
+ master to a standalone MySQL server acting as the slave, or
+ from a standalone MySQL master server to to a slave cluster;
+ in either of these cases, the standalone MySQL server uses a
+ storage engine other than <literal>NDBCLUSTER</literal>.
+ Multi-master replication setups such as circular replication
+ are also supported.
</para>
</formalpara>
@@ -96,12 +212,13 @@
<listitem>
<formalpara>
- <title>Support for disk-based records</title>
+ <title>Support for storage of rows on disk</title>
<para>
- Records on disk are now supported. Indexed columns,
- including the primary key hash index, must still be stored
- in RAM; however, all other columns can be stored on disk.
+ Storage of <literal>NDBCLUSTER</literal> table data on disk
+ is now supported. Indexed columns, including the primary key
+ hash index, must still be stored in RAM; however, all other
+ columns can be stored on disk.
</para>
</formalpara>
@@ -114,16 +231,17 @@
<listitem>
<formalpara>
- <title>Variable-sized records</title>
+ <title>Variable-size columns</title>
<para>
- A column defined as <literal>VARCHAR(255)</literal>
- currently uses 260 bytes of storage independent of what is
- stored in any particular record. In MySQL 5.1 Cluster
- tables, only the portion of the column actually taken up by
- the record will be stored. This will make possible a
- reduction in space requirements for such columns by a factor
- of 5 in many cases.
+ In MySQL 5.0, an <literal>NDBCLUSTER</literal> table column
+ defined as <literal>VARCHAR(255)</literal> used 260 bytes of
+ storage independent of what was stored in any particular
+ record. In MySQL 5.1 Cluster tables, only the portion of the
+ column actually taken up by the record is stored. This makes
+ possible a significant reduction in space requirements for
+ such columns as compared to previous release series —
+ by a factor of up to 5 in many cases.
</para>
</formalpara>
@@ -157,8 +275,10 @@
<para>
The MySQL Server can also determine whether it is possible to
<quote>prune away</quote> some of the partitions from the
- <literal>WHERE</literal> clause. See
- <xref linkend="partitioning-pruning"/>.
+ <literal>WHERE</literal> clause, which can greatly speed up
+ some queries. See <xref linkend="partitioning-pruning"/>, for
+ information about designing tables and queries to take
+ advantage of partition pruning.
</para>
</listitem>
@@ -168,25 +288,104 @@
<title>Autodiscovery of table schema changes</title>
<para>
- In MySQL 5.1, you no longer need to issue <literal>FLUSH
- TABLES</literal> or a <quote>dummy</quote>
+ In MySQL 5.0, it was necessary to issue a <literal>FLUSH
+ TABLES</literal> statement or a <quote>dummy</quote>
<literal>SELECT</literal> in order for new
- <literal>NDB</literal> tables or changes made to schemas of
- existing <literal>NDB</literal> tables on one SQL node to be
- visible on the cluster's other SQL nodes.
+ <literal>NDBCLUSTER</literal> tables or changes made to
+ schemas of existing <literal>NDBCLUSTER</literal> tables on
+ one SQL node to be visible on the cluster's other SQL
+ nodes. In MySQL 5.1, this is no longer necessary; new
+ Cluster tables and changes in the definitions of existing
+ <literal>NDBCLUSTER</literal> tables made on one SQL node
+ are immediately visible to all SQL nodes connected to the
+ cluster.
</para>
</formalpara>
<note>
<para>
- When creating a new database, it is still necessary to issue
- a <literal>CREATE DATABASE</literal> or <literal>CREATE
- SCHEMA</literal> statement on each SQL node in the cluster.
+ When creating a new database, it is still necessary in MySQL
+ 5.1 to issue a <literal>CREATE DATABASE</literal> or
+ <literal>CREATE SCHEMA</literal> statement on each SQL node
+ in the cluster.
</para>
</note>
</listitem>
+ <listitem>
+ <formalpara>
+
+ <title>Distribution awareness (NDB API)</title>
+
+ <para id="mysql-cluster-changes-5-1-distribution-awareness">
+ <firstterm>Distribution awareness</firstterm> is a mechanism
+ by which the best data node is automatically selected to be
+ queried for information. (Conceptually, it is similar in
+ some ways to partition pruning (see
+ <xref linkend="partitioning-pruning"/>). To take advantage
+ of distribution awareness, you should do the following:
+
+ <orderedlist>
+
+ <listitem>
+ <para>
+ Determine which table column is most likely to be used
+ for finding matching records.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Make this column part of the table's primary key.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Explicitly partition the table by
+ <literal>KEY</literal>, using this column as the
+ table' partitioning key.
+ </para>
+ </listitem>
+
+ </orderedlist>
+
+ Following these steps causes records with the same value for
+ the partitioning column to be stored on the same partition
+ (that is, in the same node group). When reading data,
+ transactions are begun on the data node actually having the
+ desired rows instead of this node being determined by the
+ usual round-robin mechanism.
+
+ <important>
+ <para>
+ In order to see a measureable impact on performance, the
+ cluster must have at least four data nodes, since, with
+ only two data nodes, both data nodes have exactly the
+ same data.
+ </para>
+ </important>
+
+ Using distribution awareness can yield performance increase
+ of as great as 45% when using four data nodes, and possibly
+ more when using a greater number of data nodes.
+
+ <note>
+ <para>
+ In mainline MySQL 5.1 releases, distribution awareness
+ was supported only when using the NDB API; support was
+ added for SQL and API nodes in &mccge-series; 6.3 (see
+ <xref linkend="mysql-cluster-changes-5-1-ndb-6-3"/>,
+ which includes an example showing how to create a table
+ in order to take advantage of distribution awareness).
+ </para>
+ </note>
+ </para>
+
+ </formalpara>
+ </listitem>
+
</itemizedlist>
<para>
@@ -215,126 +414,97 @@
</section>
-<section id="mysql-cluster-changes-5-1-ndb-6-1">
+ <section id="mysql-cluster-changes-5-1-ndb-6-1">
- <title>Overview of MySQL Cluster Changes in &mccge-series;
6.1.x</title>
+ <title>Features Added in &mccge-series; 6.1</title>
<para>
The following list provides an overview of significant feature
- additions and changes made in &mccge-series; 6.1.x releases. For
- detailed information, see
- <xref linkend="mysql-cluster-news-6-1"/>.
+ additions and changes made in &mccge-series; 6.1. All of the
+ changes in this list are also available in &mccge-series; 6.2 and
+ 6.3 releases. For detailed information about all changes made in
+ &mccge-series; 6.1, see <xref linkend="mysql-cluster-news-6-1"/>.
<itemizedlist>
<listitem>
- <para>
- Enhanced backup status reporting, aided in part by the
- introduction of a <literal>BackupReportFrequency</literal>
- configuration parameter (MySQL 5.1.14-ndb-6.1.0).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- Maximum number of all nodes in a cluster increased to 255
- (MySQL 5.1.14-ndb-6.1.1).
- </para>
- </listitem>
+ <title>Increased number of cluster nodes</title>
- <listitem>
- <para>
- Addition of the <literal>NDB</literal> API
- <literal>Dictionary::listEvents()</literal> method (MySQL
- 5.1.15-ndb-6.1.3).
- </para>
- </listitem>
+ <para>
+ The maximum number of all nodes in a MySQL Cluster has
+ been increased to 255. For more information, see
+ <xref linkend="mysql-cluster-limitations-multiple-nodes"/>.
+ </para>
- <listitem>
- <para>
- Ability to disable arbitration, by setting
- <literal>ArbitrationRank=0</literal> on all nodes (MySQL
- 5.1.15-ndb-6.1.3).
- </para>
+ </formalpara>
</listitem>
<listitem>
- <para>
- Inclusion of the <command>ndbd_redo_log_reader</command>
- utility in the default build (MySQL 5.1.15-ndb-6.1.3).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- New methods of the <literal>Ndb_cluster_connection</literal>
- class, making it possible to iterate over all existing
- <literal>Ndb</literal> objects (MySQL 5.1.15-ndb-6.1.4).
- </para>
- </listitem>
+ <title>Disabling arbitration</title>
- <listitem>
- <para>
- <option>--ndb-wait-connected</option> option for
- <command>mysqld</command>, causing
<command>mysqld</command>
- wait a specified amount of time to be connected to the
- cluster before starting to accept client connections (MySQL
- 5.1.15-ndb-6.1.4).
- </para>
- </listitem>
+ <para>
+ It is now possible to disable arbitration by setting
+ <literal>ArbitrationRank=0</literal> on all cluster
+ management and SQL nodes. For more information, see
+ <link
linkend="mysql-cluster-param-mgm-definition-arbitrationrank"><citetitle>Defining
+ the Management Server:
+
<literal>ArbitrationRank</literal></citetitle></link> and
+ <link
linkend="mysql-cluster-param-api-definition-arbitrationrank"><citetitle>Defining
+ SQL and Other API Nodes:
+
<literal>ArbitrationRank</literal></citetitle></link>.
+ </para>
- <listitem>
- <para>
- Improved data node memory allocation (MySQL
- 5.1.15-ndb-6.1.4).
- </para>
+ </formalpara>
</listitem>
<listitem>
- <para>
- Ability to pipe output of <command>ndb_restore</command> to
- CSV file (MySQL 5.1.15-ndb-6.1.5).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- A new <literal>FragmentLogFileSize</literal> configuration
- parameter makes it possible to set the size of redo log
- files (MySQL 5.1.15-ndb-6.1.11).
- </para>
+ <title>Additional <literal>DUMP</literal>
commands</title>
+
+ <para>
+ New management client <literal>DUMP</literal> commands
+ provide help with tracking transactions, scan operations,
+ and locks. See
+ <xref linkend="mysql-cluster-mgm-client-commands"/>, and
+ <xref linkend="ndb-internals-dump-commands"/>, for more
+ information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- <literal>MaxAllocate</literal> configuration parameter makes
- it possible to set the maximum size of the allocation unit
- used for table memory (MySQL 5.1.15-ndb-6.1.12).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- New management client <literal>DUMP</literal> commands
- providing help with tracking transactions, scan operations,
- and locks (MySQL 5.1.15-ndb-6.1.12).
- </para>
+ <title>Faster Disk Data backups</title>
+
+ <para>
+ Improvements in backups of Disk Data tables can yield a 10
+ to 15% increase in backup speed of Disk Data tables.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Improvements in backups of Disk Data tables resulted in a 10
- to 15% increase in backup speed of Disk Data tables (MySQL
- 5.1.15-ndb-6.1.13).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- Batching of updates on cluster replication slaves, enabled
- using the <option>--slave-allow-batching</option> option for
- <command>mysqld</command> (MySQL 5.1.15-ndb-6.1.17).
- </para>
+ <title>Batched slave updates</title>
+
+ <para>
+ Batching of updates on cluster replication slaves, enabled
+ using the <option>--slave-allow-batching</option> option
+ for <command>mysqld</command>, increases replication
+ efficiency. For more information, see
+ <xref linkend="mysql-cluster-replication-starting"/>.
+ </para>
+
+ </formalpara>
</listitem>
</itemizedlist>
@@ -344,213 +514,281 @@
<section id="mysql-cluster-changes-5-1-ndb-6-2">
- <title>Overview of MySQL Cluster Changes in &mccge-series;
6.2.x</title>
+ <title>Features Added in &mccge-series; 6.2</title>
<para>
The following list provides an overview of significant feature
- additions and changes made in &mccge-series; 6.2.x releases. For
- detailed information, see
+ additions and changes made in &mccge-series; 6.2. All of the
+ changes in this list are also available in &mccge-series; 6.3 .
+ For more detailed information about all feature changes and
+ bugfixes made in &mccge-series; 6.2, see
<xref linkend="mysql-cluster-news-6-2"/>.
<itemizedlist>
<listitem>
- <para>
- Mutliple cluster connections by a single MySQL server using
- the <option>--ndb-cluster-connection-pool</option> startup
- option for <command>mysqld</command> (MySQL
- 5.1.18-ndb-6.2.2).
- </para>
+ <formalpara>
+
+ <title>Enhanced backup status reporting</title>
+
+ <para>
+ Backup status reporting has been improved, aided in part
+ by the introduction of a
+ <literal>BackupReportFrequency</literal> configuration
+ parameter; see
+ <link
linkend="mysql-cluster-param-ndbd-definition-backupreportfrequency"><citetitle>Defining
+ Data Nodes:
+
<literal>BackupReportFrequency</literal></citetitle></link>,
+ for more information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- New management client <literal>DUMP</literal> commands
- providing help with tracking transactions, scan operations,
- and locks (MySQL 5.1.18-ndb-6.2.2).
- </para>
+ <formalpara>
+
+ <title>Multiple cluster connections per SQL node</title>
+
+ <para>
+ A single MySQL server acting as a MySQL Cluster SQL node
+ can employ multiple connections to the cluster using the
+ <option>--ndb-cluster-connection-pool</option> startup
+ option for <command>mysqld</command>. This option is
+ described in
+ <link
linkend="option_mysqld_ndb-cluster-connection-pool"><citetitle>MySQL
+ Cluster-Related Command Options for
+ <command>mysqld</command>:
+ <option>--ndb-cluster-connection-pool</option>
+ option</citetitle></link>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- The addition of the <literal>NdbRecord</literal> interface
- and handler for the <literal>NDB</literal> API (MySQL
- 5.1.19-ndb-6.2.3).
- </para>
+ <formalpara>
+
+ <title>New data access interface</title>
+
+ <para>
+ The <literal>NdbRecord</literal> interface provides a new
+ and simplified data handler for use in NDB API
+ applications. See <xref linkend="interface-ndbrecord"/>,
+ for more information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Enhanced reporting, aided in part by the introduction of a
- <literal>BackupReportFrequency</literal> configuration
- parameter as well as new management client <literal>REPORT
- BackupStatus</literal> and <literal>REPORT
- MemoryUsage</literal> commands (MySQL 5.1.19-ndb-6.2.3).
- </para>
+ <formalpara>
+
+ <title>New reporting commands</title>
+
+ <para>
+ The new management client <literal>REPORT
+ BackupStatus</literal> and <literal>REPORT
+ MemoryUsage</literal> commands provide better access to
+ information about the status of MySQL Cluster backups and
+ how much memory is being used by MySQL Cluster for data
+ and index storage. See
+ <xref linkend="mysql-cluster-mgm-client-commands"/>, for
+ more information about the <literal>REPORT</literal>
+ commands. In addition, in-progress status reporting is
+ provided by the <command>ndb_restore</command> utility;
+ see <xref linkend="mysql-cluster-restore"/>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- In-progress status reporting by
- <command>ndb_restore</command> (MySQL 5.1.19-ndb-6.2.3).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- Batching of updates on cluster replication slaves, enabled
- using the <option>--slave-allow-batching</option> option for
- <command>mysqld</command> (MySQL 5.1.19-ndb-6.2.3).
- </para>
+ <title>Improved memory allocation and configuration</title>
+
+ <para>
+ Memory is now allocated by the <literal>NDB</literal>
+ kernel to tables on a page-by-page basis, which
+ significantly reduces the memory overhead required for
+ maintaining <literal>NDBCLUSTER</literal> tables. In
+ addition, the <literal>MaxAllocate</literal> configuration
+ parameter now makes it possible to set the maximum size of
+ the allocation unit used for table memory; for more
+ information about this configuratin parameter, see
+ <link
linkend="mysql-cluster-param-ndbd-definition-maxallocate"><citetitle>Defining
+ Data Nodes:
+ <literal>MaxAllocate</literal></citetitle></link>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Improved memory allocation in the <literal>NDB</literal>
- kernel (MySQL 5.1.19-ndb-6.2.3).
- </para>
+ <para></para>
</listitem>
<listitem>
- <para>
- Improvements in backups of Disk Data tables resulted in a 10
- to 15% increase in backup speed of Disk Data tables (MySQL
- 5.1.19-ndb-6.2.3).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- <literal>MaxAllocate</literal> configuration parameter makes
- it possible to set the maximum size of the allocation unit
- used for table memory (MySQL 5.1.19-ndb-6.2.3).
- </para>
+ <title>Choice of fixed-width or variable-width columns</title>
+
+ <para>
+ You can control whether fixed-width or variable-width
+ storage is used for a given column of an
+ <literal>NDB</literal> table by employing of the
+ <literal>COLUMN_FORMAT</literal> specifier as part of the
+ column's definition in a <literal>CREATE
+ TABLE</literal> or <literal>ALTER TABLE</literal>
+ statement. In addition, the ability to control whether a
+ given column of an <literal>NDB</literal> table is stored
+ in memory or on disk, using the <literal>STORAGE</literal>
+ specifier as part of the column's definition in a
+ <literal>CREATE TABLE</literal> or <literal>ALTER
+ TABLE</literal> statement. For more information, see
+ <xref linkend="create-table"/>, and
+ <xref linkend="alter-table"/>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- The ability to control whether fixed-width or variable-width
- storage is used for a given column of an
- <literal>NDB</literal> table by means of the
- <literal>COLUMN_FORMAT</literal> specifier as part of the
- column's definition in a <literal>CREATE TABLE</literal> or
- <literal>ALTER TABLE</literal> statement. In addition, the
- ability to control whether a given column of an
- <literal>NDB</literal> table is stored in memory or on disk,
- using the <literal>STORAGE</literal> specifier as part of
- the column's definition in a <literal>CREATE TABLE</literal>
- or <literal>ALTER TABLE</literal> statement. (MySQL
- 5.1.19-ndb-6.2.5)
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- The <option>--bind-address</option> cluster management
- server startup option makes it possible to restrict
- management client connections to <command>ndb_mgmd</command>
- to a single host (IP address or hostname) and port. (MySQL
- 5.1.19-ndb-6.2.5)
- </para>
+ <title>Controlling management client connections</title>
+
+ <para>
+ The <option>--bind-address</option> cluster management
+ server startup option makes it possible to restrict
+ management client connections to
+ <command>ndb_mgmd</command> to a single host (IP address
+ or hostname) and port, which can make MySQL Cluster
+ management operations more secure. For more information
+ about this option, see
+ <xref linkend="mysql-cluster-ndb-mgmd-command-options"/>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Due to a change in the protocol for handling of global
- checkpoints (GCPs handled in this manner sometimes being
- referred to as <quote>micro-GCPs</quote>), it is now
- possible to control how often the GCI number is updated, and
- how often global checkpoints are written to disk, using the
- <literal>TimeBetweenEpochs</literal> configuration
- parameter. This improves the reliability and performance of
- MySQL Cluster Replication. (MySQL 5.1.22-ndb-6.2.5)
- </para>
+ <formalpara>
- <para>
- Additional fine-tuning of micro-GCPs is made possible using
- the <literal>TimeBetweenEpochsTimeout</literal> confiuration
- parameter. (MySQL 5.1.22-ndb-6.2.7)
- </para>
+ <title>Micro-GCPs</title>
+
+ <para>
+ Due to a change in the protocol for handling of global
+ checkpoints (GCPs handled in this manner sometimes being
+ referred to as <quote>micro-GCPs</quote>), it is now
+ possible to control how often the GCI number is updated,
+ and how often global checkpoints are written to disk,
+ using the <literal>TimeBetweenEpochs</literal>
+ configuration parameter. This improves the reliability and
+ performance of MySQL Cluster Replication. For more
+ information, see
+ <link
linkend="mysql-cluster-param-ndbd-definition-timebetweenepochs"><citetitle>Defining
+ Data Nodes:
+
<literal>TimeBetweenEpochs</literal></citetitle></link>
+ and
+ <link
linkend="mysql-cluster-param-ndbd-definition-timebetweenepochstimeout"><citetitle>Defining
+ Data Nodes:
+
<literal>TimeBetweenEpochsTimeout</literal></citetitle></link>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Support for the online <literal>ALTER TABLE</literal>
- operations <literal>ADD COLUMN</literal>, <literal>ADD
- INDEX</literal>, and <literal>DROP INDEX</literal> is
- available. When the <literal>ONLINE</literal> keyword is
- used, the <literal>ALTER TABLE</literal> is non-copying,
- which means that indexes do not have to be re-created, which
- provides these benefits:
+ <formalpara>
- <itemizedlist>
+ <title>Core online schema change support</title>
- <listitem>
- <para>
- Single user mode is no longer required for
- <literal>ALTER TABLE</literal> operations that can be
- performed online.
- </para>
- </listitem>
+ <para>
+ Support for the online <literal>ALTER TABLE</literal>
+ operations <literal>ADD COLUMN</literal>, <literal>ADD
+ INDEX</literal>, and <literal>DROP INDEX</literal> is
+ available. When the <literal>ONLINE</literal> keyword is
+ used, the <literal>ALTER TABLE</literal> is non-copying,
+ which means that indexes do not have to be re-created,
+ which provides these benefits:
- <listitem>
- <para>
- Transactions can continue during <literal>ALTER
- TABLE</literal> operations that can be performed
- online.
- </para>
- </listitem>
+ <itemizedlist>
- <listitem>
- <para>
- Tables being altered online are not locked.
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Single user mode is no longer required for
+ <literal>ALTER TABLE</literal> operations that can
+ be performed online.
+ </para>
+ </listitem>
- </itemizedlist>
+ <listitem>
+ <para>
+ Transactions can continue during <literal>ALTER
+ TABLE</literal> operations that can be performed
+ online.
+ </para>
+ </listitem>
- Online <literal>CREATE INDEX</literal> and <literal>DROP
- INDEX</literal> statements are also supported. Online
- changes can be suppressed using the
- <literal>OFFLINE</literal> key word. See
- <xref linkend="alter-table"/>,
- <xref linkend="create-index"/>, and
- <xref linkend="drop-index"/>, for more detailed information.
- (MySQL 5.1.22-ndb-6.2.5)
- </para>
+ <listitem>
+ <para>
+ Tables being altered online are not locked.
+ </para>
+ </listitem>
- <para>
- Additional checks against unsupported
- <literal>ONLINE</literal> operations were implemented, and
- unnecessary checks were removed. (MySQL 5.1.22-ndb-6.2.7)
- </para>
+ </itemizedlist>
+
+ Online <literal>CREATE INDEX</literal> and <literal>DROP
+ INDEX</literal> statements are also supported. Online
+ changes can be suppressed using the
+ <literal>OFFLINE</literal> key word. See
+ <xref linkend="alter-table"/>,
+ <xref linkend="create-index"/>, and
+ <xref linkend="drop-index"/>, for more detailed
+ information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- More information has been added to the
- <literal>mysql.ndb_binlog_index</literal> table so that it
- is possible to determine which originating epochs have been
- applied inside an epoch. This is useful in 3-way
- replication. (MySQL 5.1.22-ndb-6.2.6)
- </para>
+ <formalpara>
+
+ <title><literal>mysql.ndb_binlog_index</literal>
improvements</title>
+
+ <para>
+ More information has been added to the
+ <literal>mysql.ndb_binlog_index</literal> table so that it
+ is possible to determine which originating epochs have
+ been applied inside an epoch. This is particularly useful
+ for 3-way replication. See
+ <xref linkend="mysql-cluster-replication-schema"/>, for
+ more information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- <literal>DUMP 8011</literal> provides subscription data in
- the cluster log. (MySQL 5.1.22-ndb-6.2.9)
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- The <literal>MaxBufferedEpochs</literal> data node
- configuration parameter provides control over the maximum
- number of unprocessed epochs by which a subscribing node can
- lag. Subscribers which exceed this number are disconnected
- and forced to reconnect. (MySQL 5.1.23-ndb-6.2.14)
- </para>
+ <title>Epoch lag control</title>
+
+ <para>
+ The <literal>MaxBufferedEpochs</literal> data node
+ configuration parameter provides a means to control the
+ maximum number of unprocessed epochs by which a
+ subscribing node can lag. Subscribers which exceed this
+ number are disconnected and forced to reconnect. For a
+ discussion of this configuration parameter, see
+ <link
linkend="mysql-cluster-param-ndbd-definition-maxbufferedepochs"><citetitle>Defining
+ Data Nodes:
+
<literal>MaxBufferedEpochs</literal></citetitle></link>.
+ </para>
+
+ </formalpara>
</listitem>
</itemizedlist>
@@ -560,211 +798,112 @@
<section id="mysql-cluster-changes-5-1-ndb-6-3">
- <title>Overview of MySQL Cluster Changes in &mccge-series;
6.3.x</title>
+ <title>Features Added in &mccge-series; 6.3</title>
<para>
The following list provides an overview of significant feature
- additions and changes made in &mccge-series; 6.3.x releases. For
- detailed information, see
+ additions and changes first made in &mccge-series; 6.3. For more
+ detailed information about all feature changes and bugfixes made
+ in &mccge-series; 6.3, see
<xref linkend="mysql-cluster-news-6-3"/>.
<itemizedlist>
<listitem>
- <para>
- Enhanced reporting, aided in part by the introduction of a
- <literal>BackupReportFrequency</literal> configuration
- parameter as well as new management client <literal>REPORT
- BackupStatus</literal> and <literal>REPORT
- MemoryUsage</literal> commands (MySQL 5.1.19-ndb-6.3.0).
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- Implementation of conflict resolution for use in
- multi-master replication (MySQL 5.1.19-ndb-6.3.0).
- </para>
+ <title>Conflict detection and resolution</title>
- <para>
- Improvements in conflict resolution for handling
- simultaneous updates (MySQL-5.1.22-ndb-6.3.4).
- </para>
- </listitem>
+ <para>
+ It is now possible to detect and resolve conflicts that
+ arise in multi-master replication scenarios, such as
+ circular replication, when different masters may try to
+ update the same row on the slave with different data. Both
+ <quote>greatest timestamp wins</quote> and <quote>same
+ timestamp wins</quote> scenarios are supported. For more
+ information, see
+ <xref linkend="mysql-cluster-replication-conflict-resolution"/>.
+ </para>
- <listitem>
- <para>
- More information has been added to the
- <literal>mysql.ndb_binlog_index</literal> table so that it
- is possible to determine which originating epochs have been
- applied inside an epoch. This is useful in 3-way
- replication. (MySQL 5.1.22-ndb-6.3.2)
- </para>
+ </formalpara>
</listitem>
<listitem>
- <para>
- The ability to control whether fixed-width or variable-width
- storage is used for a given column of an
- <literal>NDB</literal> table by means of the
- <literal>COLUMN_FORMAT</literal> specifier as part of the
- column's definition in a <literal>CREATE TABLE</literal> or
- <literal>ALTER TABLE</literal> statement. In addition, the
- ability to control whether a given column of an
- <literal>NDB</literal> table is stored in memory or on disk,
- using the <literal>STORAGE</literal> specifier as part of
- the column's definition in a <literal>CREATE TABLE</literal>
- or <literal>ALTER TABLE</literal> statement. (MySQL
- 5.1.22-ndb-6.3.2)
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- The <option>--bind-address</option> cluster management
- server startup option makes it possible to restrict
- management client connections to <command>ndb_mgmd</command>
- to a single host (IP address or hostname) and port. (MySQL
- 5.1.22-ndb-6.3.2)
- </para>
- </listitem>
+ <title>Recovery of <quote>one master, many slaves</quote>
replication setups</title>
- <listitem>
- <para>
- Due to a change in the protocol for handling of global
- checkpoints (GCPs handled in this manner sometimes being
- referred to as <quote>micro-GCPs</quote>), it is now
- possible to control how often the GCI number is updated, and
- how often global checkpoints are written to disk, using the
- <literal>TimeBetweenEpochs</literal> configuration
- parameter. This improves the reliability and performance of
- MySQL Cluster Replication. (MySQL 5.1.22-ndb-6.3.2)
- </para>
+ <para>
+ Recovery of multi-way replication setups (<quote>one
+ master, many slaves</quote>) is now supported via the
+ <option>--ndb-log-orig</option> server option and changes
+ in the <literal>mysql.ndb_binlog_index</literal> table.
+ See <xref linkend="mysql-cluster-replication-schema"/>,
+ for more information.
+ </para>
- <para>
- Additional fine-tuning of micro-GCPs is made possible using
- the <literal>TimeBetweenEpochsTimeout</literal> confiuration
- parameter. (MySQL 5.1.22-ndb-6.3.4)
- </para>
+ </formalpara>
</listitem>
<listitem>
- <para>
- Support for the online <literal>ALTER TABLE</literal>
- operations <literal>ADD COLUMN</literal>, <literal>ADD
- INDEX</literal>, and <literal>DROP INDEX</literal> is
- available. When the <literal>ONLINE</literal> keyword is
- used, the <literal>ALTER TABLE</literal> is non-copying,
- which means that indexes do not have to be re-created, which
- provides these benefits:
+ <formalpara>
- <itemizedlist>
+ <title>Enhanced selection options for transaction
coordinator</title>
- <listitem>
- <para>
- Single user mode is no longer required for
- <literal>ALTER TABLE</literal> operations that can be
- performed online.
- </para>
- </listitem>
+ <para>
+ New values and behaviors are introduced for
+ <option>--ndb_optimized_node_selection</option> allowing
+ for greater flexibility when an SQL node chooses a
+ transaction coordinator. See
+ <link
linkend="option_mysqld_ndb_optimized_node_selection"><citetitle>MySQL
+ Cluster System Variables:
+
<literal>ndb_optimized_node_selection</literal></citetitle></link>,
+ for more information.
+ </para>
- <listitem>
- <para>
- Transactions can continue during <literal>ALTER
- TABLE</literal> operations that can be performed
- online.
- </para>
- </listitem>
+ </formalpara>
+ </listitem>
- <listitem>
- <para>
- Tables being altered online are not locked.
- </para>
- </listitem>
+ <listitem>
+ <formalpara>
- </itemizedlist>
+ <title>Replication heartbeats</title>
- Online <literal>CREATE INDEX</literal> and <literal>DROP
- INDEX</literal> statements are also supported. Online
- changes can be suppressed using the
- <literal>OFFLINE</literal> key word. See
- <xref linkend="alter-table"/>,
- <xref linkend="create-index"/>, and
- <xref linkend="drop-index"/>, for more detailed information.
- (MySQL 5.1.22-ndb-6.3.2)
- </para>
- </listitem>
+ <para>
+ Replication heartbeats facilitate the task of monitoring
+ and detecting failures in master-slave connections in real
+ time. This feature is implemented via a new
+ <literal>MASTER_HEARTBEAT_PERIOD =
+ <replaceable>value</replaceable></literal> clause for the
+ <literal>CHANGE MASTER TO</literal> statement and the
+ addition of two status variables
+ <literal>Slave_heartbeat_period</literal> and
+ <literal>Slave_received_heartbeats</literal>. For more
+ information, see <xref linkend="change-master-to"/>.
+ </para>
- <listitem>
- <para>
- Recovery of multi-way replication setups (one master, many
- slaves) is now supported via the
- <option>--ndb-log-orig</option> server option and changes in
- the <literal>mysql.ndb_binlog_index</literal> table. (MySQL
- 5.1.22-ndb-6.3.2)
- </para>
+ </formalpara>
</listitem>
<listitem>
- <para>
- New values and behaviors are introduced for
- <option>--ndb_optimized_node_selection</option> allowing for
- greater flexibility when an SQL node chooses a transaction
- coordinator. (MySQL 5.1.22-ndb-6.3.4)
- </para>
- </listitem>
+ <formalpara>
- <listitem>
- <para>
- A <option>--bind-address</option> option has been added to a
- number of MySQL client programs for use on computers having
- multiple network interfaces. The option allows you to choose
- which interface (IP address or hostname) is used to connect
- to the MySQL server. (MySQL 5.1.22-ndb-6.3.4)
- </para>
- </listitem>
+ <title><literal>NDB</literal> thread locks</title>
- <listitem>
- <para>
- A <option>--master-bind</option> option has been added to
- <command>mysqld</command>. This makes it possible to specify
- the network interface to use for connecting to the master by
- a replication slave having multiple network addresses. This
- can also be set at run time using the <literal>MASTER_BIND =
- '<replaceable>interface</replaceable>'</literal> clause in
a
- <literal>CHANGE MASTER TO</literal> statement.(MySQL
- 5.1.22-ndb-6.3.4)
- </para>
- </listitem>
+ <para>
+ It is possible to lock <literal>NDB</literal> execution
+ threads and maintenance threads (such as filesystem and
+ other operating system threads) to specific CPUs on
+ multiprocessor data node hosts, and to leverage real-time
+ scheduling using configuration parameters introduced in
+ MySQL 5.1.22-ndb-6.3.4.
+ </para>
- <listitem>
- <para>
- Replication heartbeats facilitate the task of monitoring and
- detecting failures in master-slave connections in real time.
- This feature is implemented via a new
- <literal>MASTER_HEARTBEAT_PERIOD =
- <replaceable>value</replaceable></literal> clause for the
- <literal>CHANGE MASTER TO</literal> statement and the
- addition of two status variables
- <literal>Slave_heartbeat_period</literal> and
- <literal>Slave_received_heartbeats</literal>. (MySQL
- 5.1.22-ndb-6.3.4)
- </para>
+ </formalpara>
</listitem>
<listitem>
<para>
- It is possible to lock <literal>NDB</literal> execution
- threads and maintenance threads (such as filesystem and
- other operating system threads) to specific CPUs on
- multiprocessor data node hosts, and to leverage real-time
- scheduling using configuration parameters introduced in
- MySQL 5.1.22-ndb-6.3.4.
- </para>
- </listitem>
-
- <listitem>
- <para>
The number of unnecessary reads when performing a primary
key or unique key update has been greatly reduced. Since it
is seldom necessary to read a record prior to an update,
@@ -776,128 +915,304 @@
</listitem>
<listitem>
- <para>
- Batched operations are better supported for
- <literal>DELETE</literal> and
<literal>UPDATE</literal>;
- <literal>UPDATE WHERE...</literal> and multiple
- <literal>DELETE</literal> operations are now correctly
- implemented. (MySQL 5.1.22-ndb-6.3.6)
- </para>
+ <formalpara>
+
+ <title>Batching improvements</title>
+
+ <para>
+ Support of batched <literal>DELETE</literal> and
+ <literal>UPDATE</literal> operations has been
+ significantly improved. Batching of <literal>UPDATE
+ WHERE...</literal> and multiple <literal>DELETE</literal>
+ operations is also now implemented.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Introduced the <literal>Ndb_execute_count</literal> system
- status variable, which measures the number of round trips
- made by SQL statements to the <literal>NDB</literal> kernel.
- (MySQL 5.1.22-ndb-6.3.6)
- </para>
+ <formalpara>
+
+ <title>Improved SQL statement performance metrics</title>
+
+ <para>
+ The <literal>Ndb_execute_count</literal> system status
+ variable measures the number of round trips made by SQL
+ statements to the <literal>NDB</literal> kernel, providing
+ an improved metric for determining efficiency with which
+ statements are excuted. For more information, see
+ <link
linkend="option_mysqld_Ndb_execute_count"><citetitle>MySQL
+ Cluster Status Variables:
+
<literal>Ndb_execute_count</literal></citetitle></link>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Compressed local checkpoints and backups can save 50% or
- more of the disk space used by uncompressed LCPs and
- backups. These can be enabled using the two new data node
- configuration parameters <literal>CompressedLCP</literal>
- and <literal>CompressedBackup</literal>, respectively.
- (MySQL 5.1.23-ndb-6.3.7)
- </para>
+ <formalpara>
+
+ <title>Compressed LCPs and backups</title>
+
+ <para>
+ Compressed local checkpoints and backups can save 50% or
+ more of the disk space used by uncompressed LCPs and
+ backups. These can be enabled using the two new data node
+ configuration parameters <literal>CompressedLCP</literal>
+ and <literal>CompressedBackup</literal>, respectively. See
+ <link
linkend="mysql-cluster-param-ndbd-definition-compressedbackup"><citetitle>MySQL
+ Cluster Status Variables:
+
<literal>CompressedBackup</literal></citetitle></link>,
+ and
+ <link
linkend="mysql-cluster-param-ndbd-definition-compressedlcp"><citetitle>MySQL
+ Cluster Status Variables:
+
<literal>CompressedLCP</literal></citetitle></link>, for
+ more information about these parameters.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- <literal>OPTIMIZE TABLE</literal> is supported for dynamic
- columns of in-memory <literal>NDB</literal> tables. In such
- cases, it is no longer necessary to drop (and possibly to
- re-create) a table, or to perform a rolling restart, in
- order to recover memory from deleted rows for general re-use
- by Cluster. The performance of <literal>OPTIMIZE</literal>
- on Cluster tables can be tuned by adjusting the value of the
- <literal>ndb_optimization_delay</literal> system variable,
- which controls the number of milliseconds to wait between
- processing batches of rows by <literal>OPTIMIZE
- TABLE</literal>. (MySQL 5.1.23-ndb-6.3.7)
- </para>
+ <formalpara>
- <para>
- <literal>OPTIMIZE TABLE</literal> can now be interrupted.
- This can be done, for example, by killing the SQL thread
- performing the <literal>OPTIMIZE</literal> operation. (MySQL
- 5.1.23-ndb-6.3.8)
- </para>
+ <title><literal>OPTIMIZE TABLE</literal> support with
+ <literal>NDBCLUSTER</literal> tables</title>
+
+ <para>
+ <literal>OPTIMIZE TABLE</literal> is supported for dynamic
+ columns of in-memory <literal>NDB</literal> tables. In
+ such cases, it is no longer necessary to drop (and
+ possibly to re-create) a table, or to perform a rolling
+ restart, in order to recover memory from deleted rows for
+ general re-use by Cluster. The performance of
+ <literal>OPTIMIZE</literal> on Cluster tables can be tuned
+ by adjusting the value of the
+ <literal>ndb_optimization_delay</literal> system variable,
+ which controls the number of milliseconds to wait between
+ processing batches of rows by <literal>OPTIMIZE
+ TABLE</literal>. In addition, <literal>OPTIMIZE
+ TABLE</literal> on an <literal>NDBCLUSTER</literal> table
+ can be interrupted by, for example, killing the SQL thread
+ performing the <literal>OPTIMIZE</literal> operation.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- It is possible to cause statements occurring within the same
- transaction to be run as a batch by setting the session
- variable <literal>transaction_allow_batching</literal> to
- <literal>1</literal> or <literal>ON</literal>. To use
this
- feature, <literal>AUTOCOMMIT</literal> must be set to
- <literal>0</literal> or <literal>OFF</literal>.
(MySQL
- 5.1.23-ndb-6.3.7)
- </para>
+ <formalpara>
- <para>
- Batch sizes can be controlled using the
- <option>--ndb-batch-size</option> option for
- <command>mysqld</command>. (MySQL 5.1.23-ndb-6.3.8)
- </para>
+ <title>Batching of transactions</title>
+
+ <para>
+ It is possible to cause statements occurring within the
+ same transaction to be run as a batch by setting the
+ session variable
+ <literal>transaction_allow_batching</literal> to
+ <literal>1</literal> or <literal>ON</literal>. To
use this
+ feature, <literal>AUTOCOMMIT</literal> must be set to
+ <literal>0</literal> or <literal>OFF</literal>.
Batch
+ sizes can be controlled using the
+ <option>--ndb-batch-size</option> option for
+ <command>mysqld</command>. For more information, see
+ <xref linkend="mysql-cluster-mysqld-command-options"/>,
+ and <xref linkend="mysql-cluster-system-variables"/>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- It is possible using <command>ndb_restore</command> to
- restore data reliably from a column of a given type to a
- column that uses a <quote>larger</quote> type. (This is also
- referred to as <firstterm>attribute promotion</firstterm>.)
- For example, MySQL Cluster backup data that originated in a
- <literal>SMALLINT</literal> column can be restored to a
- <literal>MEDIUMINT</literal>, <literal>INT</literal>,
or
- <literal>BIGINT</literal> column. (MySQL 5.1.23-ndb-6.3.8)
- </para>
+ <formalpara>
+
+ <title>Attribute promotion with
<command>ndb_restore</command></title>
+
+ <para>
+ It is possible using <command>ndb_restore</command> to
+ restore data reliably from a column of a given type to a
+ column that uses a <quote>larger</quote> type. This is
+ sometimes referred to as <firstterm>attribute
+ promotion</firstterm>. For example, MySQL Cluster backup
+ data that originated in a <literal>SMALLINT</literal>
+ column can be restored to a <literal>MEDIUMINT</literal>,
+ <literal>INT</literal>, or
<literal>BIGINT</literal>
+ column. See <xref linkend="mysql-cluster-restore"/>, for
+ more information.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- <literal>NDB_LE_MemoryUsage.page_size_kb</literal> has been
- renamed to
- <literal>NDB_LE_MemoryUsage.page_size_bytes</literal>.
- (MySQL 5.1.23-ndb-6.3.8)
- </para>
+ <formalpara>
+
+ <title>Parallel data node recovery</title>
+
+ <para>
+ Recovery of multiple data nodes can now be done in
+ parallel, rather than sequentially. In other words,
+ several data nodes can be restored concurrently, which can
+ often result in much faster recovery times than when they
+ are restored one at a time.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Recovery of multiple data nodes can be done in parallel,
- rather than sequentially, which can result in much faster
- recovery times. (MySQL 5.1.23-ndb-6.3.8)
- </para>
+ <formalpara>
+
+ <title>Increased local checkpoint efficiency</title>
+
+ <para>
+ Only 2 local checkpoints are stored, rather than 3,
+ lowering disk space requirements and the size and number
+ of redo log files.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Only 2 local checkpoints are stored, rather than 3, lowering
- disk space requirements and the size and number of redo log
- files. (MySQL 5.1.23-ndb-6.3.8)
- </para>
+ <formalpara>
+
+ <title><literal>NDBCLUSTER</literal> table persistence
control</title>
+
+ <para>
+ Persistence of <literal>NDB</literal> tables can be
+ controlled using the session variables
+ <literal>ndb_table_temporary</literal> and
+ <literal>ndb_table_no_logging</literal>.
+ <literal>ndb_table_no_logging</literal> causes
+ <literal>NDB</literal> tables not to be checkpointed to
+ disk; <literal>ndb_table_temporary</literal> does the
+ same, and in addition, no schema files are created. See
+ <xref linkend="mysql-cluster-option-tables"/>.
+ </para>
+
+ </formalpara>
</listitem>
<listitem>
- <para>
- Persistence of <literal>NDB</literal> tables can be
- controlled using the session variables
- <literal>ndb_table_temporary</literal> and
- <literal>ndb_table_no_logging</literal>.
- <literal>ndb_table_no_logging</literal> causes
- <literal>NDB</literal> tables not to be checkpointed to
- disk; <literal>ndb_table_temporary</literal> does the same,
- and in addition, no schema files are created. (MySQL
- 5.1.23-ndb-6.3.8)
- </para>
+ <formalpara>
+
+ <title>Epoll support (<emphasis>Linux
only</emphasis>)</title>
+
+ <para>
+ <firstterm>Epoll</firstterm> is an improved method for
+ handling file descriptors, which is more efficient than
+ scanning to determine whether a file descriptor has data
+ to be read. (The term <firstterm>epoll</firstterm> is
+ specific to Linux and equivalent functionality is known by
+ other names on other platforms such as Solaris and
+ FreeBSD.) Currently, MySQL Cluster supports this
+ functionality on Linux only.
+ </para>
+
+ </formalpara>
</listitem>
+ <listitem>
+ <formalpara>
+
+ <title>Distribution awareness (SQL nodes)</title>
+
+ <para>
+ In &mccge-series; 6.3, SQL nodes can take advantage of
+ <link
linkend="mysql-cluster-changes-5-1-distribution-awareness">distribution
+ awareness</link>. Here we provide a brief example showing
+ how to design a table to make a given class of queries
+ distrubtion-aware. Suppose an
+ <literal>NDBCLUSTER</literal> table
<literal>t1</literal>
+ has the following schema:
+
+<programlisting>
+CREATE TABLE t1 (
+ userid INT NOT NULL,
+ serviceid INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
+ data VARCHAR(255)
+) ENGINE=NDBCLUSTER;
+</programlisting>
+
+ Suppose further that most of the queries to be used in our
+ application test values of the <literal>userid</literal>
+ column of this table. The form of such a query looks
+ something like this:
+
+<programlisting>
+SELECT <replaceable>columns</replaceable> FROM t1
+ WHERE userid <replaceable>relation</replaceable>
<replaceable>value</replaceable>;
+</programlisting>
+
+ In this query, <replaceable>relation</replaceable>
+ represents some relational operator, such as
+ <literal>=</literal>, <literal><</literal>,
+ <literal>></literal>, and so on. Queries using
+ <literal>IN</literal> and a list of values can also be
+ used:
+
+<programlisting>
+SELECT <replaceable>columns</replaceable> FROM t1
+ WHERE userid IN <replaceable>value_list</replaceable>;
+</programlisting>
+
+ In order to make use of distribution awareness, we need to
+ make the <literal>userid</literal> column part of the
+ table's primary key, then explicitly partition the
+ table with this column being used as the partitioning key.
+ (Recall that for a partitioned table having one or more
+ unique keys, all columns of the table's partitioning
+ key must also be part of all of the unique keys —
+ for more information and examples, see
+ <xref
linkend="partitioning-limitations-partitioning-keys-unique-keys"/>.)
+ In other words, the table schema should be equivalent to
+ the following <literal>CREATE TABLE</literal> statement:
+
+<programlisting>
+CREATE TABLE t1 (
+ userid INT NOT NULL,
+ serviceid INT NOT NULL AUTO_INCREMENT,
+ data VARCHAR(255),
+ PRIMARY KEY p (userid,serviceid)
+) ENGINE=NDBCLUSTER
+ PARTITION BY KEY(userid);
+</programlisting>
+
+ When the table is partitioned in this way, all rows having
+ the same <literal>userid</literal> value are found on the
+ same node group, and the MySQL Server can immediately
+ select the optimal node to use as the transaction
+ coordinator.
+ </para>
+
+ </formalpara>
+ </listitem>
+
+ <listitem>
+ <formalpara>
+
+ <title>Realtime extensions for multiple CPUs</title>
+
+ <para>
+ When running MySQL Cluster data nodes on hosts with
+ multiple processors, the realtime extensions make it
+ possible to give priority to the data node process and
+ control on which CPU cores it should operate. This can be
+ done using the data node configuration parameters
+ <literal>RealtimeScheduler</literal>,
+ <literal>SchedulerExecutionTimer</literal> and
+ <literal>SchedulerSpinTimer</literal>. Doing so properly
+ can significantly lower response times and make them much
+ more predictable response. For more information about
+ using these parameters, see
+ <link
linkend="mysql-cluster-realtime-performance-parameters">Defining
+ Data Nodes: Realtime Performance Parameters</link>
+ </para>
+
+ </formalpara>
+ </listitem>
+
</itemizedlist>
</para>
Modified: trunk/pt/refman-5.1/Makefile.depends
===================================================================
--- trunk/pt/refman-5.1/Makefile.depends 2008-05-21 12:39:50 UTC (rev 10798)
+++ trunk/pt/refman-5.1/Makefile.depends 2008-05-21 15:12:31 UTC (rev 10799)
Changed blocks: 3, Lines Added: 9, Lines Deleted: 0; 1882 bytes
@@ -1733,6 +1733,7 @@
../../ndbapi/metadata/class-ndboperation.idmap \
../../ndbapi/metadata/class-ndbtransaction.idmap \
../../ndbapi/metadata/class-table.idmap \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
../../ndbapi/metadata/mgm-api.idmap \
../../ndbapi/metadata/ndb-errors.idmap \
../../ndbapi/metadata/ndb-internals.idmap \
@@ -2194,9 +2195,16 @@
mysql_cluster_roadmap_IMAGES =
mysql_cluster_roadmap_SOURCES = mysql-cluster-roadmap.xml
$(mysql_cluster_roadmap_INCLUDES)
mysql_cluster_roadmap_IDMAPS = \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
+ ../../ndbapi/metadata/ndb-internals.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-backup.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-configuration.idmap \
../../refman-5.1/metadata/mysql-cluster-disk-data.idmap \
../../refman-5.1/metadata/mysql-cluster-limitations.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-management.idmap \
../../refman-5.1/metadata/mysql-cluster-news-core.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-optvar-core.idmap \
+ ../../refman-5.1/metadata/mysql-cluster-process-management.idmap \
../../refman-5.1/metadata/mysql-cluster-replication.idmap \
../../refman-5.1/metadata/mysql-cluster-roadmap.idmap \
../../refman-5.1/metadata/partitioning.idmap \
@@ -2357,6 +2365,7 @@
../../ndbapi/metadata/class-dictionary.idmap \
../../ndbapi/metadata/class-ndb-cluster-connection.idmap \
../../ndbapi/metadata/class-ndb.idmap \
+ ../../ndbapi/metadata/interface-ndbrecord.idmap \
../../ndbapi/metadata/mgm-api.idmap \
../../ndbapi/metadata/ndb-errors.idmap \
../../ndbapi/metadata/ndb-internals.idmap \
Modified: trunk/pt/refman-5.1/mysql-cluster-roadmap.xml
===================================================================
--- trunk/pt/refman-5.1/mysql-cluster-roadmap.xml 2008-05-21 12:39:50 UTC (rev 10798)
+++ trunk/pt/refman-5.1/mysql-cluster-roadmap.xml 2008-05-21 15:12:31 UTC (rev 10799)
Changed blocks: 12, Lines Added: 866, Lines Deleted: 551; 66769 bytes
@@ -19,11 +19,6 @@
<primary>New features in MySQL Cluster</primary>
</indexterm>
- <remark role="todo">
- [js] Update to get rid of items already implemented that were listed
- as "future" items here. (Waiting on feedback from Jeb, et al.)
- </remark>
-
<remark role="note">
[js] Altered previously version-specific section ID, title.
</remark>
@@ -31,35 +26,143 @@
<remark>
Author: Jon Stephens. Based on Mikael Ronström's "Cluster News"
announcement of 2005-04-06 and other info from Mikael Ronström,
- Jonas Oreland, et al.
+ Jonas Oreland, et al. Updated with information from NDB developers
+ and support staff; May 2008 reorganisation inspired and assisted by
+ Johan Andersson.
</remark>
- <remark role="todo">
- [pd] I see nothing for next-series. [js] Commented out reference for
- now.
- </remark>
-
<para>
In this section, we discuss changes in the implementation of MySQL
Cluster in MySQL ¤t-series; and &mccge-series; as compared to
MySQL &previous-series;.
+ </para>
-<!--
- We also discuss our roadmap for further improvements to MySQL Cluster
- as currently planned for MySQL &next-series;.
+<!--
+ <para>
+
+ <remark role="TODO">
+ [js] Uncomment and expand this para when we have some likely future
+ improvements to describe, or possibly add new section for these.
+ </remark>
+ We also discuss our roadmap for further improvements to MySQL Cluster
+ as currently planned for MySQL &next-series;.
+ </para>
-->
- </para>
<para>
There are a number of significant changes in the implementation of
- the <literal>NDBCLUSTER</literal> storage engine in MySQL 5.1 as
- compared to that in MySQL 5.0. For an overview of these changes, see
- <xref linkend="mysql-cluster-changes-5-1"/>
+ the <literal>NDBCLUSTER</literal> storage engine in mainline MySQL
+ 5.1 releases up to and including MySQL 5.1.23 as compared to that in
+ MySQL 5.0; &mccge-series; makes further changes and improvements in
+ MySQL Cluster in addition to these. The changes and features most
+ likely to be of interest are shown in the following table:
+
+ <remark role="TODO">
+ [js] Add IDs to items in subsections and make entries in this
+ table links to these...
+ </remark>
+
+ <informaltable>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link linkend="mysql-cluster-changes-5-1">MySQL 5.1
(through
+ 5.1.23)</link></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>MySQL Cluster Replication</entry>
+ </row>
+ <row>
+ <entry>Disk Data storage</entry>
+ </row>
+ <row>
+ <entry>Variable-size columns</entry>
+ </row>
+ <row>
+ <entry>User-defined partitioning</entry>
+ </row>
+ <row>
+ <entry>Autodiscovery of table schema changes</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ <tgroup cols="1">
+ <colspec colwidth="50*"/>
+ <thead>
+ <row>
+ <entry><link
linkend="mysql-cluster-changes-5-1-ndb-6-1">&mccge-series;
+ 6.1</link></entry>
