Author: jstephens
Date: 2009-02-04 00:25:32 +0100 (Wed, 04 Feb 2009)
New Revision: 13546
Log:
Fixes Docs Bug #42477 (unclear docs -- tablespaces & data files)
Completes WL#3218 (Using Symbolic Links with Cluster Disk Data Tables)
Other improvements to section
Modified:
trunk/refman-5.1/mysql-cluster-disk-data.xml
Modified: trunk/refman-5.1/mysql-cluster-disk-data.xml
===================================================================
--- trunk/refman-5.1/mysql-cluster-disk-data.xml 2009-02-03 21:20:25 UTC (rev 13545)
+++ trunk/refman-5.1/mysql-cluster-disk-data.xml 2009-02-03 23:25:32 UTC (rev 13546)
Changed blocks: 15, Lines Added: 283, Lines Deleted: 180; 24427 bytes
@@ -51,12 +51,63 @@
<title>MySQL Cluster Disk Data Objects</title>
<para>
- This section discusses Disk Data objects — which include
- tables, log file groups, and tablespaces — as well as how to
- create and drop them.
+ MySQL Cluster Disk Data storage is implemented using a number of
+ <firstterm>Disk Data objects</firstterm>. These include the
+ following:
</para>
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ <firstterm>Tablespaces</firstterm> act as containers for other
+ Disk Data objects.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <firstterm>Undo log files</firstterm> undo information
+ required for rolling back transactions.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ One or more undo log files are assigned to a <firstterm>log
+ file group</firstterm>, which is then assigned to a
+ tablespace.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <firstterm>Data files</firstterm> store Disk Data table data.
+ A data file is assigned directly to a tablespace.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
<para>
+ Undo log files and data files are actual files in the filesystem
+ of each data node; by default they are placed in
+ <literal>ndb_<replaceable>node_id</replaceable>_fs</literal> in
+ the <replaceable>DataDir</replaceable> specified in the MySQL
+ Cluster <filename>config.ini</filename> file, and where
+ <replaceable>node_id</replaceable> is the data node's node
+ ID. It is possible to place these elsewhere by specifying either
+ an absolute or relative path as part of the filename when creating
+ the undo log or data file. Statements that create these files are
+ shown later in this section.
+ </para>
+
+ <para>
+ MySQL Cluster tablespaces and log file groups are not implemented
+ as files.
+ </para>
+
+ <para>
Assuming that you have already set up a MySQL Cluster with all
nodes (including management and SQL nodes) running MySQL 5.1.6 or
newer, the basic steps for creating a Cluster table on disk are as
@@ -72,24 +123,25 @@
<listitem>
<para>
- Create a <firstterm>log file group</firstterm>, and assign one
- or more undo log files to it (an undo log file is also
- referred as an <firstterm>undofile</firstterm>).
+ Create a log file group, and assign one or more undo log files
+ to it (an undo log file is also sometimes referred to as an
+ <firstterm>undofile</firstterm>).
</para>
<note>
<para>
- In MySQL 5.1, undo log files are necessary only for Disk
- Data tables. They are no longer used for tables that are
- stored in memory.
+ In MySQL 5.1 and MySQL Cluster NDB 6.x, undo log files are
+ necessary only for Disk Data tables. They are no longer used
+ for <literal role="se">NDBCLUSTER</literal> tables that are
+ stored only in memory.
</para>
</note>
</listitem>
<listitem>
<para>
- Create a <firstterm>tablespace</firstterm>, and assign the log
- file group to it, as well as one or more data files.
+ Create a tablespace; assign the log file group, as well as one
+ or more data files, to the tablespace.
</para>
</listitem>
@@ -103,8 +155,9 @@
</orderedlist>
<para>
- Each of these tasks can be accomplished using SQL statements, as
- shown in the following example.
+ Each of these tasks can be accomplished using SQL statements in
+ the <command>mysql</command> client or other MySQL client
+ application, as shown in the example that follows.
</para>
<orderedlist>
@@ -119,37 +172,37 @@
We create a log file group named <literal>lg_1</literal> using
<literal>CREATE LOGFILE GROUP</literal>. This log file group
is to be made up of two undo log files, which we name
- <filename>undo_1.dat</filename> and
- <filename>undo_2.dat</filename>, whose initial sizes are 16 MB
+ <filename>undo_1.log</filename> and
+ <filename>undo_2.log</filename>, whose initial sizes are 16 MB
and 12 MB, respectively. (The default initial size for an undo
log file is 128 MB.) Optionally, you can also specify a size
- for the log file group's <literal>UNDO</literal> buffer, or
- allow it to assume the default value of 8 MB. In this example,
- we set the UNDO buffer's size at 2 MB. A log file group must
- be created with an undo log file; so we add
- <filename>undo_1.dat</filename> to <literal>lg_1</literal> in
- this <literal>CREATE LOGFILE GROUP</literal> statement:
+ for the log file group's undo buffer, or allow it to assume
+ the default value of 8 MB. In this example, we set the UNDO
+ buffer's size at 2 MB. A log file group must be created with
+ an undo log file; so we add <filename>undo_1.log</filename> to
+ <literal>lg_1</literal> in this <literal>CREATE LOGFILE
+ GROUP</literal> statement:
</para>
<programlisting>
CREATE LOGFILE GROUP lg_1
- ADD UNDOFILE 'undo_1.dat'
+ ADD UNDOFILE 'undo_1.log'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE 2M
- ENGINE NDB;
+ ENGINE NDBCLUSTER;
</programlisting>
<para>
- To add <filename>undo_2.dat</filename> to the log file group,
+ To add <filename>undo_2.log</filename> to the log file group,
use the following <literal>ALTER LOGFILE GROUP</literal>
statement:
</para>
<programlisting>
ALTER LOGFILE GROUP lg_1
- ADD UNDOFILE 'undo_2.dat'
+ ADD UNDOFILE 'undo_2.log'
INITIAL_SIZE 12M
- ENGINE NDB;
+ ENGINE NDBCLUSTER;
</programlisting>
<para>
@@ -160,9 +213,9 @@
<listitem>
<para>
- The <filename>.dat</filename> file extension used here is
- not required. We use it merely to make the log and data
- files easily recognisable.
+ The <filename>.log</filename> file extension used here is
+ not required. We use it merely to make the log files
+ easily recognisable.
</para>
</listitem>
@@ -172,14 +225,14 @@
<literal>ALTER LOGFILE GROUP</literal> statement must
include an <literal>ENGINE</literal> clause. In MySQL
¤t-series;, the permitted values for this clause are
- <literal role="se">NDB</literal> and
- <literal role="se">NDBCLUSTER</literal>.
+ <literal role="se">NDBCLUSTER</literal> and
+ <literal role="se">NDB</literal>.
</para>
<important>
<para>
In MySQL 5.1.8 and later, there can exist only one log
- file group at any given time.
+ file group in the same MySQL Cluster at any given time.
</para>
</important>
</listitem>
@@ -191,11 +244,18 @@
'<replaceable>filename</replaceable>'</literal>, a file
with the name <replaceable>filename</replaceable> is
created in the
- <literal>ndb_<replaceable>nodeid</replaceable>_fs</literal>
- directory within the <literal>DataDirectory</literal> of
- each data node in the cluster, where
- <replaceable>nodeid</replaceable> is the node ID of the
- data node.
+ <literal>ndb_<replaceable>node_id</replaceable>_fs</literal>
+ directory within the <literal>DataDir</literal> of each
+ data node in the cluster, where
+ <replaceable>node_id</replaceable> is the node ID of the
+ data node. Each undo log file is of the size specified in
+ the SQL statement. For example, if a MySQL Cluster has 4
+ data nodes, then the <literal>ALTER LOGFILE
+ GROUP</literal> statement just shown creates 4 undo log
+ files, 1 each on in the data directory of each of the 4
+ data nodes; each of these files is named
+ <filename>undo_2.log</filename> and each file is 12 MB in
+ size.
</para>
</listitem>
@@ -253,12 +313,12 @@
ADD DATAFILE 'data_1.dat'
USE LOGFILE GROUP lg_1
INITIAL_SIZE 32M
- ENGINE NDB;
+ ENGINE NDBCLUSTER;
ALTER TABLESPACE ts_1
ADD DATAFILE 'data_2.dat'
INITIAL_SIZE 48M
- ENGINE NDB;
+ ENGINE NDBCLUSTER;
</programlisting>
<para>
@@ -279,25 +339,49 @@
<listitem>
<para>
- As is the case with the file names used here for undo log
- files, there is no special significance for the
+ As is the case with the <filename>.log</filename> file
+ extension used in this example for undo log files, there
+ is no special significance for the
<filename>.dat</filename> file extension; it is used
- merely for easy recognition.
+ merely for easy recognition of data files.
</para>
</listitem>
<listitem>
<para>
+ When you add a data file to a tablespace using
+ <literal>ADD DATAFILE
+ '<replaceable>filename</replaceable>'</literal>, a file
+ with the name <replaceable>filename</replaceable> is
+ created in the
+ <literal>ndb_<replaceable>node_id</replaceable>_fs</literal>
+ directory within the <literal>DataDir</literal> of each
+ data node in the cluster, where
+ <replaceable>node_id</replaceable> is the node ID of the
+ data node. Each undo log file is of the size specified in
+ the SQL statement. For example, if a MySQL Cluster has 4
+ data nodes, then the <literal>ALTER TABLESPACE</literal>
+ statement just shown creates 4 undo log files, 1 each on
+ in the data directory of each of the 4 data nodes; each of
+ these files is named <filename>data_2.dat</filename> and
+ each file is 48 MB in size.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
All <literal>CREATE TABLESPACE</literal> and
<literal>ALTER TABLESPACE</literal> statements must
contain an <literal>ENGINE</literal> clause; only tables
using the same storage engine as the tablespace can be
created in the tablespace. In MySQL ¤t-series;, the
only permitted values for this clause are
- <literal role="se">NDB</literal> and
- <literal role="se">NDBCLUSTER</literal>.
+ <literal role="se">NDBCLUSTER</literal> and
+ <literal role="se">NDB</literal>.
</para>
+ </listitem>
+ <listitem>
<para>
For more information about the <literal>CREATE
TABLESPACE</literal> and <literal>ALTER
@@ -331,11 +415,11 @@
INDEX(last_name, first_name)
)
TABLESPACE ts_1 STORAGE DISK
- ENGINE NDB;
+ ENGINE NDBCLUSTER;
</programlisting>
<para>
- The <literal>TABLESPACE ... STORAGE DISK</literal> clause
+ The <literal>TABLESPACE ... STORAGE DISK</literal> option
tells the <literal role="se">NDBCLUSTER</literal> storage
engine to use tablespace <literal>ts_1</literal> for disk data
storage.
@@ -399,13 +483,9 @@
You may use absolute and relative file system paths with
<literal>ADD UNDOFILE</literal> and <literal>ADD
DATAFILE</literal>. Relative paths are calculated relative to the
- data node's data directory.
-
- <remark role="note">
- [js] Or maybe not... Have asked Jeb to resolve this issue.
- </remark>
-
-<!--You may also use symbolic links.-->
+ data node's data directory. You may also use symbolic links; see
+ <xref linkend="mysql-cluster-disk-data-symlinks"/>, for more
+ information and examples.
</para>
<para>
@@ -423,7 +503,7 @@
<listitem>
<para>
- A log file group cannot be dropped, so long as any tablespaces
+ A log file group cannot be dropped as long as any tablespaces
are using it.
</para>
</listitem>
@@ -462,17 +542,17 @@
mysql> <userinput>ALTER TABLESPACE ts_1</userinput>
-> <userinput>DROP DATAFILE 'data_2.dat'</userinput>
- -> <userinput>ENGINE NDB;</userinput>
+ -> <userinput>ENGINE NDBCLUSTER;</userinput>
mysql> <userinput>ALTER TABLESPACE ts_1</userinput>
-> <userinput>DROP DATAFILE 'data_1.dat'</userinput>
- -> <userinput>ENGINE NDB;</userinput>
+ -> <userinput>ENGINE NDBCLUSTER;</userinput>
mysql> <userinput>DROP TABLESPACE ts_1</userinput>
- -> <userinput>ENGINE NDB;</userinput>
+ -> <userinput>ENGINE NDBCLUSTER;</userinput>
mysql> <userinput>DROP LOGFILE GROUP lg_1</userinput>
- -> <userinput>ENGINE NDB;</userinput>
+ -> <userinput>ENGINE NDBCLUSTER;</userinput>
</programlisting>
<para>
@@ -493,144 +573,112 @@
</section>
-<!--<section id="mysql-cluster-disk-data-symlinks">
-
- <title>Using Symbolic Links with Disk Data Objects</title>
-
- <remark role="note">
- [js] Thanks to Jeb Miller for his help with this section!
- </remark>
-
- <para>
- The performance of a MySQL Cluster that uses Disk Data storage
- can be greatly improved by separating data node file systems
- from undo log files and tablespace data files and placing these
- on different disks. While there is currently no direct support
- for this in MySQL Cluster, it is possible to achieve this
- separation using symbolic links.
- </para>
-
- <para>
- Each data node in the cluster creates a file system in the
- directory named
- <filename>ndb_<replaceable>node_id</replaceable>_fs</filename>
- under the data node's
- <literal><link linkend="mysql-cluster-param-ndbd-definition-datadir">DataDir</link></literal>
- as defined in the <filename>config.ini</filename> file. In this
- example, we assume that each data node host has 3 disks, aliased
- as <filename>/data0</filename>, <filename>/data1</filename>, and
- <filename>/data2</filename>, and that the cluster's
- <filename>config.ini</filename> includes the following:
-
- <programlisting>
+ <section id="mysql-cluster-disk-data-symlinks">
+
+ <title>Using Symbolic Links with Disk Data Objects</title>
+
+ <remark role="note">
+ [js] Thanks to Jeb Miller for his help with this section!
+ </remark>
+
+ <para>
+ The performance of a MySQL Cluster that uses Disk Data storage can
+ be greatly improved by separating data node file systems from undo
+ log files and tablespace data files and placing these on different
+ disks. While there is currently no direct support for this in
+ MySQL Cluster, it is possible to achieve this separation using
+ symbolic links.
+ </para>
+
+ <para>
+ Each data node in the cluster creates a file system in the
+ directory named
+ <filename>ndb_<replaceable>node_id</replaceable>_fs</filename>
+ under the data node's
+ <literal><link linkend="mysql-cluster-param-ndbd-definition-datadir">DataDir</link></literal>
+ as defined in the <filename>config.ini</filename> file. In this
+ example, we assume that each data node host has 3 disks, aliased
+ as <filename>/data0</filename>, <filename>/data1</filename>, and
+ <filename>/data2</filename>, and that the cluster's
+ <filename>config.ini</filename> includes the following:
+
+<programlisting>
[ndbd default]
DataDir= /data0
</programlisting>
-
- Our objective is to place all Disk Data log files in
- <filename>/data1</filename>, and all Disk Data data files in
- <filename>/data2</filename>, on each data node host.
+
+ Our objective is to place all Disk Data log files in
+ <filename>/data1</filename>, and all Disk Data data files in
+ <filename>/data2</filename>, on each data node host.
+ </para>
+
+ <note>
+ <para>
+ In this example, we assume that the cluster's data node
+ hosts are all using Linux operating systems. For other
+ platforms, you may need to substitute you operating
+ system's commands for those shown here.
</para>
-
- <note>
- <para>
- In this example, we assume that the cluster's data node hosts
- are all running Linux. For other platoforms, you may need to
- substitute you operating system's commands for those shown
- here.
- </para>
- </note>
-
- <para>
- To accomplish this, perform the following steps:
-
- <itemizedlist>
-
- <listitem>
- <para>
- Under the data node file system create symbolic links
- pointing to the other drives:
-
- <programlisting>
+ </note>
+
+ <para>
+ To accomplish this, perform the following steps:
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Under the data node file system create symbolic links
+ pointing to the other drives:
+
+<programlisting>
shell> <userinput>cd /data0/ndb_2_fs</userinput>
shell> <userinput>ls</userinput>
D1 D10 D11 D2 D8 D9 LCP
shell> <userinput>ln -s /data0 dnlogs</userinput>
shell> <userinput>ln -s /data1 dndata</userinput>
</programlisting>
-
- You should now have two symbolic links:
-
- <programlisting>
-shell> <userinput>ls -l &ddash;hide=D*</userinput>
+
+ You should now have two symbolic links:
+
+<programlisting>
+shell> <userinput>ls -l --hide=D*</userinput>
lrwxrwxrwx 1 user group 30 2007-03-19 13:58 dndata -> /data1
lrwxrwxrwx 1 user group 30 2007-03-19 13:59 dnlogs -> /data2
</programlisting>
-
- We show this only for the data node with node ID 2;
- however, you must do this for <emphasis>each</emphasis>
- data node.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Now, in the <command>mysql</command> client, create a log
- file group and tablespace using the symbolic links, as
- shown here:
-
- <programlisting>
+
+ We show this only for the data node with node ID 2; however,
+ you must do this for <emphasis>each</emphasis> data node.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Now, in the <command>mysql</command> client, create a log
+ file group and tablespace using the symbolic links, as shown
+ here:
+ </para>
+
+<programlisting>
mysql> <userinput>CREATE LOGFILE GROUP lg1</userinput>
- -> <userinput>ADD UNDOFILE './dnlogs/undo1.dat'</userinput>
+ -> <userinput>ADD UNDOFILE 'dnlogs/undo1.log'</userinput>
-> <userinput>INITIAL_SIZE 150M</userinput>
-> <userinput>UNDO_BUFFER_SIZE = 1M</userinput>
- -> <userinput>ENGINE=NDB;</userinput>
+ -> <userinput>ENGINE=NDBCLUSTER;</userinput>
mysql> <userinput>CREATE TABLESPACE ts1</userinput>
- -> <userinput>ADD DATAFILE './dndata/data1.dat'</userinput>
+ -> <userinput>ADD DATAFILE 'dndata/data1.log'</userinput>
-> <userinput>USE LOGFILE GROUP lg1</userinput>
-> <userinput>INITIAL_SIZE 1G</userinput>
- -> <userinput>ENGINE=NDB;</userinput>
+ -> <userinput>ENGINE=NDBCLUSTER;</userinput>
</programlisting>
-
- <remark role="note">
- [js] Commenting this part out for now since we don't
- support more than one data node per host.
- </remark>
-
- -->
-<!--
- 4) Doing the same for multi data nodes running on one host.
-
- Give each data node its own path
- [ndbd]
- Id: 2
- HostName: host13
- FileSystemPath: /data0
-
- [ndbd]
- Id: 3
- HostName: host13
- FileSystemPath: /data1
-
- DN ID 2 will create it file system on /data0 and DN ID 3 will create its file
- system on /data1
-
- we then repeat the steps in step 2 placing the files in the desired location
- using symbolic links, and the step 3 for the actual create.
- -->
+ <para>
+ Verify that the files were created and placed correctly as
+ shown here:
+ </para>
-<!--
- </para>
- </listitem>
-
- <listitem>
- <para>
- Verify that the files were created and placed correctly as
- shown here:
-
- <programlisting>
+<programlisting>
shell> <userinput>cd /data1</userinput>
shell> <userinput>ls -l</userinput>
total 2099304
@@ -641,14 +689,69 @@
total 2099304
-rw-rw-r&ddash; 1 user group 1073741824 2007-03-19 14:02 data1.dat
</programlisting>
- </para>
- </listitem>
-
- </itemizedlist>
- </para>
-
+ </listitem>
+
+ <listitem>
+ <para>
+ If you are running multiple data nodes on one host, you must
+ take care to avoid having them try to use the same space for
+ Disk Data files. You can make this easier by creating a
+ symbolic link in each data node filesystem. Suppose you are
+ using <filename>/data0</filename> for both data node
+ filesystems, but you wish to have the Disk Data files for
+ both nodes on <filename>/data1</filename>. In this case, you
+ can do something similar to what is shown here:
+ </para>
+
+<programlisting>
+shell> <userinput>cd /data0</userinput>
+shell> <userinput>ln -s ndb_2_fs/dd /data1/dn2</userinput>
+shell> <userinput>ln -s ndb_3_fs/dd /data1/dn3</userinput>
+shell> <userinput>ls -l --hide=D* ndb_2_fs</userinput>
+lrwxrwxrwx 1 user group 30 2007-03-19 14:22 dd -> /data1/dn2
+shell> <userinput>ls -l --hide=D* ndb_3_fs</userinput>
+lrwxrwxrwx 1 user group 30 2007-03-19 14:22 dd -> /data1/dn3
+</programlisting>
+
+ <para>
+ Now you can create a logfile group and tablespace using the
+ symbolic link, like this:
+ </para>
+
+<programlisting>
+mysql> <userinput>CREATE LOGFILE GROUP lg1</userinput>
+ -> <userinput>ADD UNDOFILE 'dd/undo1.log'</userinput>
+ -> <userinput>INITIAL_SIZE 150M</userinput>
+ -> <userinput>UNDO_BUFFER_SIZE = 1M</userinput>
+ -> <userinput>ENGINE=NDBCLUSTER;</userinput>
+
+mysql> <userinput>CREATE TABLESPACE ts1</userinput>
+ -> <userinput>ADD DATAFILE 'dd/data1.log'</userinput>
+ -> <userinput>USE LOGFILE GROUP lg1</userinput>
+ -> <userinput>INITIAL_SIZE 1G</userinput>
+ -> <userinput>ENGINE=NDBCLUSTER;</userinput>
+</programlisting>
+
+ <para>
+ Verify that the files were created and placed correctly as
+ shown here:
+ </para>
+
+<programlisting>
+shell> <userinput>cd /data1</userinput>
+shell> <userinput>ls</userinput>
+dn2 dn3
+shell> <userinput>ls dn2</userinput>
+undo1.log data1.log
+shell> <userinput>ls dn3</userinput>
+undo1.log data1.log
+</programlisting>
+ </listitem>
+
+ </itemizedlist>
+ </para>
+
</section>
--->
<section id="mysql-cluster-disk-data-storage-requirements">
| Thread |
|---|
| • svn commit - mysqldoc@docsrva: r13546 - trunk/refman-5.1 | jon.stephens | 4 Feb |
