Author: mcbrown
Date: 2006-06-14 16:20:36 +0200 (Wed, 14 Jun 2006)
New Revision: 2375
Log:
Some formatting/content fixes.
Incorporating the Stored Procedure/Prepared Statement documentation from Konstantin Osipov. Thanks Konstantin!
Modified:
trunk/internals/internals.xml
Modified: trunk/internals/internals.xml
===================================================================
--- trunk/internals/internals.xml 2006-06-14 09:05:51 UTC (rev 2374)
+++ trunk/internals/internals.xml 2006-06-14 14:20:36 UTC (rev 2375)
@@ -75,135 +75,148 @@
source code. We'll close off with a few pictures of file formats.
</para>
- <para>
- <emphasis role="bold">BitKeeper</emphasis>
- </para>
+ <section id="guided-tour-bitkeeper">
- <para>
- We want to download the latest, the very latest, version of the
- MySQL server. So we won't click <quote>Downloads</quote> on the
- MySQL Developer Zone page — that's usually a few weeks old.
- Instead we'll get BitKeeper (tm), which is a revision control
- package, vaguely like CVS or Perforce. This is what MySQL's
- developers use every day, so what we download with BitKeeper is
- usually less than a day old. If you've ever submitted a bug report
- and gotten the response <quote>thanks, we fixed the bug in the
- source code repository</quote> that means you can get the fixed
- version with BitKeeper.
- </para>
+ <title>BitKeeper</title>
- <para>
- First, log on to www.bitkeeper.com and register. One way is:
- </para>
+ <para>
+ We want to download the latest, the very latest, version of the
+ MySQL server. So we won't click <quote>Downloads</quote> on the
+ MySQL Developer Zone page — that's usually a few weeks
+ old. Instead we'll get BitKeeper (tm), which is a revision
+ control package, vaguely like CVS or Perforce. This is what
+ MySQL's developers use every day, so what we download with
+ BitKeeper is usually less than a day old. If you've ever
+ submitted a bug report and gotten the response <quote>thanks, we
+ fixed the bug in the source code repository</quote> that means
+ you can get the fixed version with BitKeeper.
+ </para>
- <itemizedlist>
+ <para>
+ First, log on to www.bitkeeper.com and register. One way is:
+ </para>
- <listitem>
- <para>
- Click <quote>Downloads</quote> which takes you to the
- downloads page
- <ulink url="http://bitkeeper.com/Products.BK_Pro.Downloads.html"
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Click <quote>Downloads</quote> which takes you to the
+ downloads page
+ <ulink url="http://bitkeeper.com/Products.BK_Pro.Downloads.html"
/>
- </para>
- </listitem>
+ </para>
+ </listitem>
- <listitem>
- <para>
- Click on <quote>Software download form</quote> which takes you
- to <ulink url="http://www.bitmover.com/cgi-bin/download.cgi"/>
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Click on <quote>Software download form</quote> which takes
+ you to
+ <ulink url="http://www.bitmover.com/cgi-bin/download.cgi"/>
+ </para>
+ </listitem>
- <listitem>
- <para>
- Fill in all the fields, including the Platform name, which was
- <quote>Linux/x86 ...</quote> for us, but you'll be okay with
- other choices. Like MySQL, BitKeeper is available on many
- platforms, so the details will vary here.
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Fill in all the fields, including the Platform name, which
+ was <quote>Linux/x86 ...</quote> for us, but you'll be okay
+ with other choices. Like MySQL, BitKeeper is available on
+ many platforms, so the details will vary here.
+ </para>
+ </listitem>
- <listitem>
- <para>
- Click <quote>Send Requests</quote>.
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Click <quote>Send Requests</quote>.
+ </para>
+ </listitem>
- </itemizedlist>
+ </itemizedlist>
- <para>
- Then you'll have to wait until BitKeeper mails you some more
- instructions, which are actually quite simple. There is no fee to
- use BitKeeper for downloads of open-source code repositories like
- MySQL's.
- </para>
+ <para>
+ Then you'll have to wait until BitKeeper mails you some more
+ instructions, which are actually quite simple. There is no fee
+ to use BitKeeper for downloads of open-source code repositories
+ like MySQL's.
+ </para>
- <para>
- After you have BitKeeper, you'll be able to clone. That is, you'll
- be able to get a copy of the source code, using a statement that
- looks like this:
- </para>
+ <para>
+ After you have BitKeeper, you'll be able to clone. That is,
+ you'll be able to get a copy of the source code, using a
+ statement that looks like this:
+ </para>
<programlisting>
shell> <userinput>bk clone</userinput>
</programlisting>
- <para>
- ... that is,
- </para>
+ <para>
+ ... that is,
+ </para>
<programlisting>
bk clone <MySQL machine:/directory name> <your directory name>
</programlisting>
- <para>
- ... that is,
- </para>
+ <para>
+ ... that is,
+ </para>
-<programlisting>
-- Start a shell
-- On the shell, make a directory named (say) mysql-5.0:
- mkdir $HOME/mysql-5.0
- cd $HOME
-- bk clone bk://mysql.bkbits.net/mysql-5.0 mysql-5.0
-</programlisting>
+ <itemizedlist>
- <para>
- (The $HOME directory is usually your personal area that you're
- allowed to write to. If that's not the case, replace $HOME with
- your personal choice whenever it appears.)
- </para>
+ <listitem>
+ <para>
+ Start a shell
+ </para>
+ </listitem>
- <para>
- There is a lot of code, so the first time you do this the download
- will take over an hour. That's if you're lucky. Glitches can
- occur, for example the 'bk' command fails due to a firewall
- restriction.
- </para>
+ <listitem>
+ <para>
+ On the shell, make a directory named (say) mysql-5.0:
+ </para>
- <para>
- If you're glitch-prone, you'll need to read the manual: Section
- 2.8.3, Installing from the Development Source Tree.
- </para>
+<programlisting>shell> mkdir $HOME/mysql-5.0
+shell>cd $HOME
+shell> bk clone bk://mysql.bkbits.net/mysql-5.0 mysql-5.0</programlisting>
+ </listitem>
- <para>
- On later occasions, you'll be doing what's called a <quote>bk
- pull</quote> instead of a <quote>bk clone</quote>, and it will go
- faster. Typically a <quote>bk pull</quote> takes 10 minutes and is
- glitch-free.
- </para>
+ </itemizedlist>
- <para>
- <emphasis role="bold">Directories, Alphabetical Order</emphasis>
- </para>
+ <para>
+ (The $HOME directory is usually your personal area that you're
+ allowed to write to. If that's not the case, replace $HOME with
+ your personal choice whenever it appears.)
+ </para>
- <para>
- After bk clone finishes and says <quote>Clone completed
- successfully</quote> you'll have 40 new sets of files on your
- computer, as you'll be able to see with <command>ls</command> or
- <command>dir</command>.
- </para>
+ <para>
+ There is a lot of code, so the first time you do this the
+ download will take over an hour. That's if you're lucky.
+ Glitches can occur, for example the 'bk' command fails due to a
+ firewall restriction.
+ </para>
+ <para>
+ If you're glitch-prone, you'll need to read the manual: Section
+ 2.8.3, Installing from the Development Source Tree.
+ </para>
+
+ <para>
+ On later occasions, you'll be doing what's called a <quote>bk
+ pull</quote> instead of a <quote>bk clone</quote>, and it will
+ go faster. Typically a <quote>bk pull</quote> takes 10 minutes
+ and is glitch-free.
+ </para>
+
+ <para>
+ <emphasis role="bold">Directories, Alphabetical Order</emphasis>
+ </para>
+
+ <para>
+ After bk clone finishes and says <quote>Clone completed
+ successfully</quote> you'll have 40 new sets of files on your
+ computer, as you'll be able to see with <command>ls</command> or
+ <command>dir</command>.
+ </para>
+
<programlisting>
BUILD
BitKeeper
@@ -247,212 +260,262 @@
zlib
</programlisting>
- <para>
- These will all be installed as directories below the $HOME
- directory. At first all these directory names might intimidate
- you, and that's natural. After all, MySQL is a big project. But
- we're here to show you that there's order in this apparent chaos.
- </para>
+ <para>
+ These will all be installed as directories below the $HOME
+ directory. At first all these directory names might intimidate
+ you, and that's natural. After all, MySQL is a big project. But
+ we're here to show you that there's order in this apparent
+ chaos.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories</emphasis>
- </para>
+ </section>
-<programlisting>
-#1 BUILD
-#2 client
-#3 Docs
-#4 myisam
-#5 mysys
-#6 sql
-#7 vio
-</programlisting>
+ <section id="guided-tour-majordir">
- <para>
- The orderly approach is to look first at the most important
- directories, then we'll look at the whole list in our second pass.
- So, first, let's look at what you'll find in just seven of the
- directories: BUILD, client, Docs, myisam, mysys, sql, and vio.
- </para>
+ <title>The Major Directories</title>
- <para>
- <emphasis role="bold">The Major Directories: #1 BUILD</emphasis>
- </para>
+ <orderedlist>
- <para>
- The first major directory we'll look at is BUILD. It actually has
- very little in it, but it's useful, because one of the first
- things you might want to do with the source code is: compile and
- link it.
- </para>
+ <listitem>
+ <para>
+ <literal>BUILD</literal>
+ </para>
+ </listitem>
- <para>
- The example command line that we could use is
- </para>
+ <listitem>
+ <para>
+ <literal>client</literal>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>Docs</literal>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>myisam</literal>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>mysys</literal>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>sql</literal>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>vio</literal>
+ </para>
+ </listitem>
+
+ </orderedlist>
+
+ <para>
+ The orderly approach is to look first at the most important
+ directories, then we'll look at the whole list in our second
+ pass. So, first, let's look at what you'll find in just seven of
+ the directories: BUILD, client, Docs, myisam, mysys, sql, and
+ vio.
+ </para>
+
+ <section id="guided-tour-majordir-build">
+
+ <title>Major Directories: BUILD</title>
+
+ <para>
+ The first major directory we'll look at is BUILD. It actually
+ has very little in it, but it's useful, because one of the
+ first things you might want to do with the source code is:
+ compile and link it.
+ </para>
+
+ <para>
+ The example command line that we could use is
+ </para>
+
<programlisting>
BUILD/compile-pentium-debug --prefix=$HOME/mysql-bin
</programlisting>
- <para>
- It invokes a batch file in the BUILD directory. When it's done,
- you'll have an executable MySQL server and client.
- </para>
+ <para>
+ It invokes a batch file in the BUILD directory. When it's
+ done, you'll have an executable MySQL server and client.
+ </para>
- <para>
- Or, um, well, maybe you won't. Sometimes people have trouble with
- this step because there's something missing in their operating
- system version, or whatever. Don't worry, it really does work, and
- there are people around who might help you if you have trouble
- with this step. Search for "build" in the archives of
- lists.mysql.com.
- </para>
+ <para>
+ Or, um, well, maybe you won't. Sometimes people have trouble
+ with this step because there's something missing in their
+ operating system version, or whatever. Don't worry, it really
+ does work, and there are people around who might help you if
+ you have trouble with this step. Search for "build" in the
+ archives of lists.mysql.com.
+ </para>
- <para>
- We, when we're done building, tend to install it with
- </para>
+ <para>
+ We, when we're done building, tend to install it with the
+ following sequence:
+ </para>
<programlisting>
-make
-make install
-$HOME/mysql-bin/bin/mysql_install_db\
+shell> make
+shell> make install
+shell> $HOME/mysql-bin/bin/mysql_install_db\
--basedir=$HOME/mysql-bin\
--datadir=$HOME/mysql-bin/var
</programlisting>
- <para>
- This puts the new MySQL installation files on
- </para>
+ <para>
+ This puts the new MySQL installation files on
+ </para>
<programlisting>
-$HOME/mysql-bin/libexec -- for the server
-$HOME/mysql-bin/bin -- for the mysql client
-$HOME/mysql-bin/var -- for the databases
+shell> $HOME/mysql-bin/libexec -- for the server
+shell> $HOME/mysql-bin/bin -- for the mysql client
+shell> $HOME/mysql-bin/var -- for the databases
</programlisting>
- <para>
- <emphasis role="bold">gdb GNU debugger</emphasis>
- </para>
+ <section id="guided-tour-major-directories-build-gdb">
- <para>
- Once you've got something that runs, you can put a debugger on it.
- We recommend use of the GNU debugger
- </para>
+ <title>gdb (GNU debugger)</title>
+ <para>
+ Once you've got something that runs, you can put a debugger
+ on it. We recommend use of the GNU debugger
+ </para>
+
<programlisting>
<ulink url="http://www.gnu.org/software/gdb/documentation/"/>
</programlisting>
- <para>
- And many developers use the graphical debugger tool DDD - Data
- Display Debugger
- </para>
+ <para>
+ And many developers use the graphical debugger tool DDD -
+ Data Display Debugger
+ </para>
<programlisting>
<ulink url="http://www.gnu.org/software/ddd/manual/"/>
</programlisting>
- <para>
- These are free and common, they're probably on your Linux system
- already.
- </para>
+ <para>
+ These are free and common, they're probably on your Linux
+ system already.
+ </para>
- <para>
- There are debuggers for Windows and other operating systems, of
- course — don't feel left out just because we're mentioning a
- Linux tool name! But it happens that we do a lot of things with
- Linux ourselves, so we happen to know what to say. To debug the
- mysqld server, say:
- </para>
+ <para>
+ There are debuggers for Windows and other operating systems,
+ of course — don't feel left out just because we're
+ mentioning a Linux tool name! But it happens that we do a
+ lot of things with Linux ourselves, so we happen to know
+ what to say. To debug the mysqld server, say:
+ </para>
<programlisting>
-ddd --gdb --args \
-$HOME/mysql-bin/libexec/mysqld \
- --basedir=$HOME/mysql-bin \
- --datadir=$HOME/mysql-bin/var\
- --skip-networking
+shell> ddd --gdb --args \
+ $HOME/mysql-bin/libexec/mysqld \
+ --basedir=$HOME/mysql-bin \
+ --datadir=$HOME/mysql-bin/var\
+ --skip-networking
</programlisting>
- <para>
- From this point on, it may be tempting to follow along through the
- rest of the "guided tour" by setting breakpoints, displaying the
- contents of variables, and watching what happens after starting a
- client from another shell. That would be more fun. But it would
- require a detour, to discuss how to use the debugger. So we'll
- plow forward, the dull way, noting what's in the directories and
- using a text editor to note what's in the individual files.
- </para>
+ <para>
+ From this point on, it may be tempting to follow along
+ through the rest of the "guided tour" by setting
+ breakpoints, displaying the contents of variables, and
+ watching what happens after starting a client from another
+ shell. That would be more fun. But it would require a
+ detour, to discuss how to use the debugger. So we'll plow
+ forward, the dull way, noting what's in the directories and
+ using a text editor to note what's in the individual files.
+ </para>
- <para>
- <emphasis>Running a test with the debugger</emphasis>
- </para>
+ </section>
- <para>
- To run a test named <literal>some.test</literal> with the debugger
- in embedded mode you could do this:
- </para>
+ <section id="guided-tour-major-directories-test">
- <orderedlist>
+ <title>Running a Test with the Debugger</title>
- <listitem>
- <para>
- Run <literal>libmysqld/examples/test_run --gdb
- some.test</literal>. This creates a
- <filename>libmysqld/examples/test-gdbinit</filename> file
- which contains the required parameters for
- <command>mysqltest</command>.
- </para>
- </listitem>
+ <para>
+ To run a test named <literal>some.test</literal> with the
+ debugger in embedded mode you could do this:
+ </para>
- <listitem>
- <para>
- Make a copy of the <filename>test-gdbinit</filename> file
- (call it, for example, <filename>some-gdbinit</filename>). The
- <filename>test-gdbinit</filename> file will be removed after
- <literal>test-run --gdb</literal> has finished.
- </para>
- </listitem>
+ <orderedlist>
- <listitem>
- <para>
- Load
- <filename>libmysqld/examples/mysqltest_embedded</filename>
- into your favorite debugger, for example: <literal>gdb
- mysqltest_embedded</literal>.
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Run <literal>libmysqld/examples/test_run --gdb
+ some.test</literal>. This creates a
+ <filename>libmysqld/examples/test-gdbinit</filename>
+ file which contains the required parameters for
+ <command>mysqltest</command>.
+ </para>
+ </listitem>
- <listitem>
- <para>
- In the debugger, for example in <literal>gdb</literal>, do:
- <literal>--sou some-gdbinit</literal>
- </para>
- </listitem>
+ <listitem>
+ <para>
+ Make a copy of the <filename>test-gdbinit</filename>
+ file (call it, for example,
+ <filename>some-gdbinit</filename>). The
+ <filename>test-gdbinit</filename> file will be removed
+ after <literal>test-run --gdb</literal> has finished.
+ </para>
+ </listitem>
- </orderedlist>
+ <listitem>
+ <para>
+ Load
+ <filename>libmysqld/examples/mysqltest_embedded</filename>
+ into your favorite debugger, for example: <literal>gdb
+ mysqltest_embedded</literal>.
+ </para>
+ </listitem>
- <para>
- Now <literal>some.test</literal> is running, and you can see if
- it's passing or not.
- </para>
+ <listitem>
+ <para>
+ In the debugger, for example in <literal>gdb</literal>,
+ do: <literal>--sou some-gdbinit</literal>
+ </para>
+ </listitem>
- <para>
- If you just want to debug some queries with the embedded server
- (not the test), it's easier to just run
- <filename>libmysqld/examples/mysql</filename>. It's the embedded
- server-based clone of the usual <command>mysql</command> tool, and
- works fine under <command>gdb</command> or whatever your favorite
- debugger is.
- </para>
+ </orderedlist>
- <para>
- <emphasis role="bold">The Major Directories: #2 client</emphasis>
- </para>
+ <para>
+ Now <literal>some.test</literal> is running, and you can see
+ if it's passing or not.
+ </para>
- <para>
- The next major directory is mysql-5.0/client.
- </para>
+ <para>
+ If you just want to debug some queries with the embedded
+ server (not the test), it's easier to just run
+ <filename>libmysqld/examples/mysql</filename>. It's the
+ embedded server-based clone of the usual
+ <command>mysql</command> tool, and works fine under
+ <command>gdb</command> or whatever your favorite debugger
+ is.
+ </para>
+ </section>
+
+ </section>
+
+ <section id="guided-tour-majordir-client">
+
+ <title>Major Directories: client</title>
+
+ <para>
+ The next major directory is mysql-5.0/client.
+ </para>
+
<programlisting>
size name comment
---- ---- -------
@@ -462,50 +525,53 @@
+ 12 more .c and .cc programs
</programlisting>
- <para>
- It has the source code of many of your familiar favorites, like
- mysql, which everybody has used to connect to the MySQL server at
- one time or another. There are other utilities too — in
- fact, you'll find the source of most client-side programs here.
- There are also programs for checking the password, and for testing
- that basic functions — such as threading or access via SSL
- — are possible.
- </para>
+ <para>
+ It has the source code of many of your familiar favorites,
+ like mysql, which everybody has used to connect to the MySQL
+ server at one time or another. There are other utilities too
+ — in fact, you'll find the source of most client-side
+ programs here. There are also programs for checking the
+ password, and for testing that basic functions — such as
+ threading or access via SSL — are possible.
+ </para>
- <para>
- You'll notice, by the way, that we're concentrating on the files
- that have extension of ".c" or ".cc". By now it's obvious that C
- is our principal language although there are some utilities
- written in Perl as well.
- </para>
+ <para>
+ You'll notice, by the way, that we're concentrating on the
+ files that have extension of ".c" or ".cc". By now it's
+ obvious that C is our principal language although there are
+ some utilities written in Perl as well.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories: #3 Docs</emphasis>
- </para>
+ </section>
- <para>
- The next major directory is mysql-5.0/Docs.
- </para>
+ <section id="guided-tour-majordir-docs">
- <para>
- With the BitKeeper downloads, /Docs is nearly empty. Binary and
- source distributions include some pre-formatted documentation
- files, such as the MySQL Reference manual in Info format (for
- Unix) or CHM format (for Windows). The <literal>mysqldoc</literal>
- documentation repository is available separately from
- <ulink url="http://dev.mysql.com/tech-resources/sources.html"/>.
- If you have Subversion, you can check out a copy of the repository
- with this command:
- </para>
+ <title>Major Directories: Docs</title>
+ <para>
+ The next major directory is mysql-5.0/Docs.
+ </para>
+
+ <para>
+ With the BitKeeper downloads, /Docs is nearly empty. Binary
+ and source distributions include some pre-formatted
+ documentation files, such as the MySQL Reference manual in
+ Info format (for Unix) or CHM format (for Windows). The
+ <literal>mysqldoc</literal> documentation repository is
+ available separately from
+ <ulink url="http://dev.mysql.com/tech-resources/sources.html"/>.
+ If you have Subversion, you can check out a copy of the
+ repository with this command:
+ </para>
+
<programlisting>
svn checkout http://svn.mysql.com/svnpublic/mysqldoc/
</programlisting>
- <para>
- Some important files in the <literal>mysqldoc</literal> repository
- are:
- </para>
+ <para>
+ Some important files in the <literal>mysqldoc</literal>
+ repository are:
+ </para>
<programlisting>
name comment
@@ -516,36 +582,38 @@
+ several more .xml files
</programlisting>
- <para>
- Our documents are written in XML using DocBook. The DocBook format
- is becoming popular among publishers, and of course there's lots
- of general documentation for it, for example at
- <ulink url="http://www.docbook.org/"/>. For our immediate purpose,
- the most interesting directory might be the
- <filename>internals</filename> directory, because it contains the
- source for the Internals Manual that you're reading now.
- </para>
+ <para>
+ Our documents are written in XML using DocBook. The DocBook
+ format is becoming popular among publishers, and of course
+ there's lots of general documentation for it, for example at
+ <ulink url="http://www.docbook.org/"/>. For our immediate
+ purpose, the most interesting directory might be the
+ <filename>internals</filename> directory, because it contains
+ the source for the Internals Manual that you're reading now.
+ </para>
- <para>
- At this moment, the Internals Manual has more than 100 pages of
- information, including some details about the formats of MySQL
- files that you won't find anywhere else, and a complete
- description of the message formats that the client and server use
- to communicate. Although it's rough and may contain errors and is
- obsolete in parts, it is a document that you must read to truly
- understand the workings of MySQL.
- </para>
+ <para>
+ At this moment, the Internals Manual has more than 100 pages
+ of information, including some details about the formats of
+ MySQL files that you won't find anywhere else, and a complete
+ description of the message formats that the client and server
+ use to communicate. Although it's rough and may contain errors
+ and is obsolete in parts, it is a document that you must read
+ to truly understand the workings of MySQL.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories: #4 myisam</emphasis>
- </para>
+ </section>
- <para>
- The next major directory is labelled myisam. We will begin by
- mentioning that myisam is one of what we call the MySQL storage
- engine directories.
- </para>
+ <section id="guided-tour-majordir-myisam">
+ <title>Major Directories: myisam</title>
+
+ <para>
+ The next major directory is labelled myisam. We will begin by
+ mentioning that myisam is one of what we call the MySQL
+ storage engine directories.
+ </para>
+
<programlisting>
The MySQL storage engine directories:
heap -- also known as 'memory'
@@ -554,28 +622,28 @@
ndb -- ndb cluster
</programlisting>
- <para>
- For example the heap directory contains the source files for the
- heap storage engine and the ndb directory contains the source
- files for the ndb storage engine.
- </para>
+ <para>
+ For example the heap directory contains the source files for
+ the heap storage engine and the ndb directory contains the
+ source files for the ndb storage engine.
+ </para>
- <para>
- But the files in those directories are mostly analogues of what's
- in the myisam directory, and the myisam directory is sort of a
- 'template'.
- </para>
+ <para>
+ But the files in those directories are mostly analogues of
+ what's in the myisam directory, and the myisam directory is
+ sort of a 'template'.
+ </para>
- <para>
- On the myisam directory, you'll find the programs that do file
- I/O. Notice that the file names begin with the letters mi, by the
- way. That stands for MyISAM, and most of the important files in
- this directory start with mi.
- </para>
+ <para>
+ On the myisam directory, you'll find the programs that do file
+ I/O. Notice that the file names begin with the letters mi, by
+ the way. That stands for MyISAM, and most of the important
+ files in this directory start with mi.
+ </para>
- <para>
- File handling programs on mysql-5.0/myisam:
- </para>
+ <para>
+ File handling programs on mysql-5.0/myisam:
+ </para>
<programlisting>
size name comment
@@ -586,9 +654,9 @@
+ more mi_*.c programs
</programlisting>
- <para>
- Row handling programs on mysql-5.0/myisam:
- </para>
+ <para>
+ Row handling programs on mysql-5.0/myisam:
+ </para>
<programlisting>
size name comment
@@ -600,17 +668,17 @@
+ more mi_*.c programs
</programlisting>
- <para>
- Drilling down a bit, you'll also find programs in the myisam
- directory that handle deleting, updating, and inserting of rows.
- The only one that's a little hard to find is the program for
- inserting rows, which we've called mi_write.c instead of
- mi_insert.c.
- </para>
+ <para>
+ Drilling down a bit, you'll also find programs in the myisam
+ directory that handle deleting, updating, and inserting of
+ rows. The only one that's a little hard to find is the program
+ for inserting rows, which we've called mi_write.c instead of
+ mi_insert.c.
+ </para>
- <para>
- Key handling programs on mysql-5.0/myisam:
- </para>
+ <para>
+ Key handling programs on mysql-5.0/myisam:
+ </para>
<programlisting>
size name comment
@@ -621,37 +689,40 @@
+ more mi_*.c programs
</programlisting>
- <para>
- The final notable group of files in the myisam directory is the
- group that handles keys in indexes.
- </para>
+ <para>
+ The final notable group of files in the myisam directory is
+ the group that handles keys in indexes.
+ </para>
- <para>
- To sum up: (1) The myisam directory is where you'll find programs
- for handling files, rows, and keys. You won't find programs for
- handling columns — we'll get to them a bit later. (2) The
- myisam directory is just one of the handler directories. The
- programs in the other storage engine directories fulfill about the
- same functions.
- </para>
+ <para>
+ To sum up: (1) The myisam directory is where you'll find
+ programs for handling files, rows, and keys. You won't find
+ programs for handling columns — we'll get to them a bit
+ later. (2) The myisam directory is just one of the handler
+ directories. The programs in the other storage engine
+ directories fulfill about the same functions.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories: #5 mysys</emphasis>
- </para>
+ </section>
- <para>
- The next major directory is labelled mysys, which stands for MySQL
- System Library. This is the toolbox directory, for example it has
- low level routines for file access. The .c files in mysys have
- procedures and functions that are handy for calling by main
- programs, for example by the programs in the myisam directory.
- There are 115 .c files in mysys, so we only can note a sampling.
- </para>
+ <section id="guided-tour-majordir-mysys">
- <para>
- Sampling of programs on mysql-5.0/mysys
- </para>
+ <title>Major Directories: mysys</title>
+ <para>
+ The next major directory is labelled mysys, which stands for
+ MySQL System Library. This is the toolbox directory, for
+ example it has low level routines for file access. The .c
+ files in mysys have procedures and functions that are handy
+ for calling by main programs, for example by the programs in
+ the myisam directory. There are 115 .c files in mysys, so we
+ only can note a sampling.
+ </para>
+
+ <para>
+ Sampling of programs on mysql-5.0/mysys
+ </para>
+
<programlisting>
size name comment
---- ---- -------
@@ -661,39 +732,43 @@
+ 112 more *.c programs
</programlisting>
- <para>
- Example one: with charset.c routines, you can change the character
- set.
- </para>
+ <para>
+ Example one: with charset.c routines, you can change the
+ character set.
+ </para>
- <para>
- Example two: mf_qsort.c contains our quicksort package.
- </para>
+ <para>
+ Example two: mf_qsort.c contains our quicksort package.
+ </para>
- <para>
- Example three: mf_tempfile.c has what's needed for maintaining
- MySQL's temporary files.
- </para>
+ <para>
+ Example three: mf_tempfile.c has what's needed for maintaining
+ MySQL's temporary files.
+ </para>
- <para>
- You can see from these examples that mysys is a hodgepodge. That's
- why we went to the trouble of producing extra documentation in
- this document to help you analyze mysys's contents.
- </para>
+ <para>
+ You can see from these examples that mysys is a hodgepodge.
+ That's why we went to the trouble of producing extra
+ documentation in this document to help you analyze mysys's
+ contents.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories: #6 sql</emphasis>
- </para>
+ </section>
- <para>
- The next major directory is mysql-5.0/sql. If you remember your
- manual, you know that you must pronounce this: ess queue ell.
- </para>
+ <section id="guided-tour-majordir-sql">
- <para>
- The "parser" programs on mysql-5.0/sql:
- </para>
+ <title>Major Directories: sql</title>
+ <para>
+ The next major directory is mysql-5.0/sql. If you remember
+ your manual, you know that you must pronounce this: ess queue
+ ell.
+ </para>
+
+ <para>
+ The "parser" programs on mysql-5.0/sql:
+ </para>
+
<programlisting>
size name comment
---- ---- -------
@@ -702,15 +777,16 @@
+ many more *.cc programs
</programlisting>
- <para>
- This is where we keep the parser. In other words, programs like
- sql_lex.cc and sql_yacc.yy are responsible for figuring out what's
- in an SQL command, and deciding what to do about it.
- </para>
+ <para>
+ This is where we keep the parser. In other words, programs
+ like sql_lex.cc and sql_yacc.yy are responsible for figuring
+ out what's in an SQL command, and deciding what to do about
+ it.
+ </para>
- <para>
- The "handler" programs on mysql-5.0/sql:
- </para>
+ <para>
+ The "handler" programs on mysql-5.0/sql:
+ </para>
<programlisting>
size name comment
@@ -724,20 +800,20 @@
215091 ha_ndbcluster.cc ndb
</programlisting>
- <para>
- This is also where we keep the handler programs. Now, you'll
- recall that the storage engine itself, for example myisam, is a
- separate directory. But here in the sql directory, we have
- programs which are responsible for determining which handler to
- call, formatting appropriate arguments, and checking results. In
- other words, the programs that begin with the letters ha are the
- handler interface programs, and there's one for each storage
- engine.
- </para>
+ <para>
+ This is also where we keep the handler programs. Now, you'll
+ recall that the storage engine itself, for example myisam, is
+ a separate directory. But here in the sql directory, we have
+ programs which are responsible for determining which handler
+ to call, formatting appropriate arguments, and checking
+ results. In other words, the programs that begin with the
+ letters ha are the handler interface programs, and there's one
+ for each storage engine.
+ </para>
- <para>
- The "statement" routines in mysql-5.0/sql:
- </para>
+ <para>
+ The "statement" routines in mysql-5.0/sql:
+ </para>
<programlisting>
size name comment
@@ -752,23 +828,23 @@
+ many more sql_*.cc programs
</programlisting>
- <para>
- Also in the sql directory, you'll find individual programs for
- handling each of the syntactical components of an SQL statement.
- These programs tend to have names beginning with sql_. So for the
- SELECT statement, check out sql_select.cc.
- </para>
+ <para>
+ Also in the sql directory, you'll find individual programs for
+ handling each of the syntactical components of an SQL
+ statement. These programs tend to have names beginning with
+ sql_. So for the SELECT statement, check out sql_select.cc.
+ </para>
- <para>
- Thus, there are "statement" routines like sql_delete.c,
- sql_load.c, and sql_help.c, which take care of the DELETE, LOAD
- DATA, and HELP statements. The file names are hints about the SQL
- statements involved.
- </para>
+ <para>
+ Thus, there are "statement" routines like sql_delete.c,
+ sql_load.c, and sql_help.c, which take care of the DELETE,
+ LOAD DATA, and HELP statements. The file names are hints about
+ the SQL statements involved.
+ </para>
- <para>
- The "statement function" routines in mysql-5.0/sql:
- </para>
+ <para>
+ The "statement function" routines in mysql-5.0/sql:
+ </para>
<programlisting>
size name comment
@@ -779,52 +855,49 @@
17669 sql_union.cc unions
</programlisting>
- <para>
- Then there are the routines for components of statements, such as
- strings, or online analytical processing which at this moment just
- means ROLLUP, or user-defined functions, or the UNION operator.
- </para>
+ <para>
+ Then there are the routines for components of statements, such
+ as strings, or online analytical processing which at this
+ moment just means ROLLUP, or user-defined functions, or the
+ UNION operator.
+ </para>
- <para>
- <emphasis role="bold">The Major Directories: #7 vio</emphasis>
- </para>
+ </section>
- <para>
- The final major directory that we'll highlight today is labelled
- vio, for "virtual I/O".
- </para>
+ <section id="guided-tour-majordir-vio">
- <para>
- The vio routines are wrappers for the various network I/O calls
- that happen with different protocols. The idea is that in the main
- modules one won't have to write separate bits of code for each
- protocol. Thus vio's purpose is somewhat like the purpose of
- Microsoft's winsock library.
- </para>
+ <title>Major Directories: vio</title>
- <para>
- And the preceding paragraph about vio is actually a quotation from
- a later section of the Internals Manual.
- </para>
+ <para>
+ The final major directory that we'll highlight is labelled
+ vio, for "virtual I/O".
+ </para>
- <para>
- (We didn't lift the quotation because of laziness, but to indicate
- the sort of information that's in the documentation.)
- </para>
+ <para>
+ The vio routines are wrappers for the various network I/O
+ calls that happen with different protocols. The idea is that
+ in the main modules one won't have to write separate bits of
+ code for each protocol. Thus vio's purpose is somewhat like
+ the purpose of Microsoft's winsock library.
+ </para>
- <para>
- Okay, that wraps up our quick look at the seven major directories.
- Just one summary chart remains to do.
- </para>
+ <para>
+ That wraps up our quick look at the seven major directories.
+ Just one summary chart remains to do.
+ </para>
- <para>
- <emphasis role="bold">The Flow</emphasis>
- </para>
+ </section>
- <para>
- This is a diagram of the flow.
- </para>
+ </section>
+ <section id="guided-tour-flow">
+
+ <title>The Flow</title>
+
+ <para>
+ This is a diagram of the flow.
+ </para>
+
<programlisting>
User enters "INSERT" statement /* client */
|
@@ -841,55 +914,57 @@
Handler stores in file /* mysys */
</programlisting>
- <para>
- The diagram is very simplified — it's a caricature that
- distorts important things, but remember that we've only discussed
- seven major directories so far: Docs, BUILD, and the five that you
- see here.
- </para>
+ <para>
+ The diagram is very simplified — it's a caricature that
+ distorts important things, but remember that we've only
+ discussed seven major directories so far: Docs, BUILD, and the
+ five that you see here.
+ </para>
- <para>
- The flow works like this:
- </para>
+ <para>
+ The flow works like this:
+ </para>
- <para>
- First, the client routines get an SQL statement from a user,
- allowing edit, performing initial checks, and so on.
- </para>
+ <para>
+ First, the client routines get an SQL statement from a user,
+ allowing edit, performing initial checks, and so on.
+ </para>
- <para>
- Then, via the vio routines, the somewhat-massaged statement goes
- off to the server.
- </para>
+ <para>
+ Then, via the vio routines, the somewhat-massaged statement goes
+ off to the server.
+ </para>
- <para>
- Next, the sql routines handle the parsing and call what's
- necessary for each individual part of the statement. Along the
- way, the sql routines will be calling the low level mysys routines
- frequently.
- </para>
+ <para>
+ Next, the sql routines handle the parsing and call what's
+ necessary for each individual part of the statement. Along the
+ way, the sql routines will be calling the low level mysys
+ routines frequently.
+ </para>
- <para>
- Finally, one of the ha (handler) programs in the sql directory
- will dispatch to an appropriate handler for storage. In this case
- we've assumed, as before, that the handler is myisam — so a
- myisam-directory program is involved. Specifically, that program
- is mi_write.c, as we mentioned earlier.
- </para>
+ <para>
+ Finally, one of the ha (handler) programs in the sql directory
+ will dispatch to an appropriate handler for storage. In this
+ case we've assumed, as before, that the handler is myisam
+ — so a myisam-directory program is involved. Specifically,
+ that program is mi_write.c, as we mentioned earlier.
+ </para>
- <para>
- Simple, eh?
- </para>
+ <para>
+ Simple, eh?
+ </para>
- <para>
- <emphasis role="bold">The Open-source Directories</emphasis>
- </para>
+ </section>
- <para>
- We're now getting into the directories which aren't "major".
- Starting with:
- </para>
+ <section id="guided-tour-osdir">
+ <title>The Open-source Directories</title>
+
+ <para>
+ We're now getting into the directories which aren't "major".
+ Starting with:
+ </para>
+
<programlisting>
dbug
pstack
@@ -898,53 +973,54 @@
zlib
</programlisting>
- <para>
- Now it's time to reveal a startling fact, which is — we
- didn't write all of the source code in all of the source code
- directories all by ourselves. This list is, in a sense, a tribute
- to the idea of open source.
- </para>
+ <para>
+ Now it's time to reveal a startling fact, which is — we
+ didn't write all of the source code in all of the source code
+ directories all by ourselves. This list is, in a sense, a
+ tribute to the idea of open source.
+ </para>
- <para>
- There's dbug, which is Fred Fish's debug library.
- </para>
+ <para>
+ There's dbug, which is Fred Fish's debug library.
+ </para>
- <para>
- There's pstack, which displays the process stack.
- </para>
+ <para>
+ There's pstack, which displays the process stack.
+ </para>
- <para>
- There's regex, which is what we use for our regular expressions
- function.
- </para>
+ <para>
+ There's regex, which is what we use for our regular expressions
+ function.
+ </para>
- <para>
- There's strings, the meaning of which is obvious.
- </para>
+ <para>
+ There's strings, the meaning of which is obvious.
+ </para>
- <para>
- There's zlib, which is for Zempel-Liv compression.
- </para>
+ <para>
+ There's zlib, which is for Zempel-Liv compression.
+ </para>
- <para>
- All of the programs in these directories were supplied by others,
- as open source code. We didn't just take them, of course. MySQL
- has checked and changed what's in these directories. But we
- acknowledge with thanks that they're the products of other
- projects, and other people's labor, and we only regret that we
- won't have time to note all the contributed or publicly available
- components of MySQL, in this manual.
- </para>
+ <para>
+ All of the programs in these directories were supplied by
+ others, as open source code. We didn't just take them, of
+ course. MySQL has checked and changed what's in these
+ directories. But we acknowledge with thanks that they're the
+ products of other projects, and other people's labor, and we
+ only regret that we won't have time to note all the contributed
+ or publicly available components of MySQL, in this manual.
+ </para>
- <para>
- <emphasis role="bold">The Internal and External Storage Engine
- Directories</emphasis>
- </para>
+ </section>
- <para>
- Continuing with our extract from the directory list ...
- </para>
+ <section id="guided-tour-sedir">
+ <title>The Internal and External Storage Engine Directories</title>
+
+ <para>
+ Continuing with our extract from the directory list ...
+ </para>
+
<programlisting>
bdb /* external */
heap
@@ -954,34 +1030,36 @@
ndb
</programlisting>
- <para>
- Let's go through the idea of storage engines once more, this time
- with a list of all the storage engines, both the ones that we
- produce, and the ones that others produce. We've already mentioned
- the internal ones — so now we'll remark on the directories
- of the two common external storage engines — BDB and
- innobase.
- </para>
+ <para>
+ Let's go through the idea of storage engines once more, this
+ time with a list of all the storage engines, both the ones that
+ we produce, and the ones that others produce. We've already
+ mentioned the internal ones — so now we'll remark on the
+ directories of the two common external storage engines —
+ BDB and innobase.
+ </para>
- <para>
- The BDB, or Berkeley Database, handler, is strictly the product of
- Sleepycat software. Sleepycat has a web page at sleepycat.com,
- which contains, among other things, documentation for their
- product. So you can download Sleepycat's own documentation of the
- source code in the BDB directory.
- </para>
+ <para>
+ The BDB, or Berkeley Database, handler, is strictly the product
+ of Sleepycat software. Sleepycat has a web page at
+ sleepycat.com, which contains, among other things, documentation
+ for their product. So you can download Sleepycat's own
+ documentation of the source code in the BDB directory.
+ </para>
- <para>
- As for the innobase handler, which many of you probably use,
- you'll be happy to know that the comments in the files are
- reasonably clear (the InnoBase Oy people are pretty strict about
- comments). There are two chapters about it in this document.
- </para>
+ <para>
+ As for the innobase handler, which many of you probably use,
+ you'll be happy to know that the comments in the files are
+ reasonably clear (the InnoBase Oy people are pretty strict about
+ comments). There are two chapters about it in this document.
+ </para>
- <para>
- <emphasis role="bold">The "OS Specific" Directories</emphasis>
- </para>
+ </section>
+ <section id="guided-tour-os-specific">
+
+ <title>The "OS Specific" Directories</title>
+
<programlisting>
netware
NEW-RPMS
@@ -989,47 +1067,51 @@
VC++Files
</programlisting>
- <para>
- A few words are in order about the directories that contain files
- which relate to a particular environment that MySQL can run in.
- </para>
+ <para>
+ A few words are in order about the directories that contain
+ files which relate to a particular environment that MySQL can
+ run in.
+ </para>
- <para>
- The netware directory contains a set of files for interfacing with
- netware, and anyone who has an involvement with NetWare knows that
- we're allied with them, and so this is one of the directories that
- represents the joint enterprise.
- </para>
+ <para>
+ The netware directory contains a set of files for interfacing
+ with netware, and anyone who has an involvement with NetWare
+ knows that we're allied with them, and so this is one of the
+ directories that represents the joint enterprise.
+ </para>
- <para>
- The NEW-RPMS directory (empty at time of writing) is for Linux,
- and the os2 directory is for OS/2.
- </para>
+ <para>
+ The NEW-RPMS directory (empty at time of writing) is for Linux,
+ and the os2 directory is for OS/2.
+ </para>
- <para>
- Finally, the VC++Files directory is for Windows. We've found that
- the majority of Windows programmers who download and build MySQL
- from source use Microsoft Visual C. In the VC++Files directory you
- will find a nearly complete replication of what's in all the other
- directories that we've discussed, except that the .c files are
- modified to account for the quirks of Microsoft tools.
- </para>
+ <para>
+ Finally, the VC++Files directory is for Windows. We've found
+ that the majority of Windows programmers who download and build
+ MySQL from source use Microsoft Visual C. In the VC++Files
+ directory you will find a nearly complete replication of what's
+ in all the other directories that we've discussed, except that
+ the .c files are modified to account for the quirks of Microsoft
+ tools.
+ </para>
- <para>
- Without endorsing by particular names, we should note that other
- compilers from other manufacturers also work.
- </para>
+ <para>
+ Without endorsing by particular names, we should note that other
+ compilers from other manufacturers also work.
+ </para>
- <para>
- <emphasis role="bold">Odds and Ends</emphasis>
- </para>
+ </section>
- <para>
- Finally, for the sake of completeness, we'll put up a list of the
- rest of the directories — those that we haven't had occasion
- to mention till now.
- </para>
+ <section id="guided-tour-oddsends">
+ <title>Odds and Ends</title>
+
+ <para>
+ Finally, for the sake of completeness, we'll put up a list of
+ the rest of the directories — those that we haven't had
+ occasion to mention till now.
+ </para>
+
<programlisting>
Source Code Administration Directories:
BitKeeper
@@ -1050,48 +1132,49 @@
tools
</programlisting>
- <para>
- You don't have to worry about the administration directories since
- they're not part of what you build.
- </para>
+ <para>
+ You don't have to worry about the administration directories
+ since they're not part of what you build.
+ </para>
- <para>
- You probably won't have to worry about the stand-alone programs
- either, since you just use them, you don't need to remake them.
- </para>
+ <para>
+ You probably won't have to worry about the stand-alone programs
+ either, since you just use them, you don't need to remake them.
+ </para>
- <para>
- There's an include directory that you SHOULD have a look at,
- because the common header files for programs from several
- directories are in here.
- </para>
+ <para>
+ There's an include directory that you SHOULD have a look at,
+ because the common header files for programs from several
+ directories are in here.
+ </para>
- <para>
- Finally, there are stand-alone utility and test programs. Strictly
- speaking they're not part of the "source code". But it's probably
- reassuring to know that there's a test suite, for instance. Part
- of the quality-assurance process is to run the scripts in the test
- suite before releasing.
- </para>
+ <para>
+ Finally, there are stand-alone utility and test programs.
+ Strictly speaking they're not part of the "source code". But
+ it's probably reassuring to know that there's a test suite, for
+ instance. Part of the quality-assurance process is to run the
+ scripts in the test suite before releasing.
+ </para>
- <para>
- And those are the last. We've now traipsed through every
- significant directory created during your download of the MySQL
- source package.
- </para>
+ <para>
+ And those are the last. We've now traipsed through every
+ significant directory created during your download of the MySQL
+ source package.
+ </para>
- <para>
- <emphasis role="bold">A Chunk of Code in
- /sql/sql_update.cc</emphasis>
- </para>
+ </section>
- <para>
- Now, having finished with our bird's eye view of the source code
- from the air, let's take the perspective of the worms on the
- ground. (Which is another name for MySQL's developer staff —
- turn on laugh track here.)
- </para>
+ <section id="guided-tour-chunk">
+ <title>A Chunk of Code in <filename>/sql/sql_update.cc</filename></title>
+
+ <para>
+ Now, having finished with our bird's eye view of the source code
+ from the air, let's take the perspective of the worms on the
+ ground. (Which is another name for MySQL's developer staff
+ — turn on laugh track here.)
+ </para>
+
<programlisting>
int mysql_update(THD *thd, ...)
{
@@ -1119,67 +1202,71 @@
}
</programlisting>
- <para>
- Here's a snippet of code from a .c file in the sql directory,
- specifically from sql_update.cc, which — as we mentioned
- earlier -- is invoked when there's an UPDATE statement to process.
- </para>
+ <para>
+ Here's a snippet of code from a .c file in the sql directory,
+ specifically from sql_update.cc, which — as we mentioned
+ earlier -- is invoked when there's an UPDATE statement to
+ process.
+ </para>
- <para>
- The entire routine has many error checks with handlers for
- improbable solutions, and showing multiple screens would be
- tedious, so we've truncated the code a lot. Where you see an
- ellipsis (three dots in a row), that means "and so on".
- </para>
+ <para>
+ The entire routine has many error checks with handlers for
+ improbable solutions, and showing multiple screens would be
+ tedious, so we've truncated the code a lot. Where you see an
+ ellipsis (three dots in a row), that means "and so on".
+ </para>
- <para>
- So, what do we learn from this snippet of code? In the first
- place, we see that it's fairly conventional C code. A brace causes
- an indentation, instructions tend to be compact with few
- unnecessary spaces, and comments are sparse.
- </para>
+ <para>
+ So, what do we learn from this snippet of code? In the first
+ place, we see that it's fairly conventional C code. A brace
+ causes an indentation, instructions tend to be compact with few
+ unnecessary spaces, and comments are sparse.
+ </para>
- <para>
- Abbreviations are common, for example thd stands for thread, you
- just have to get used to them. Typically a structure will be in a
- separate .h file.
- </para>
+ <para>
+ Abbreviations are common, for example thd stands for thread, you
+ just have to get used to them. Typically a structure will be in
+ a separate .h file.
+ </para>
- <para>
- Routine names are sometimes long enough that they explain
- themselves. For example, you can probably guess that this routine
- is opening and locking, allocating memory in a cache, initializing
- a process for reading records, reading records in a loop until the
- thread is killed or there are no more to read, storing a modified
- record for the table, and — after the loop is through
- — possibly writing to the log. Incidentally, a transactional
- table is usually a BDB or an InnoDB table.
- </para>
+ <para>
+ Routine names are sometimes long enough that they explain
+ themselves. For example, you can probably guess that this
+ routine is opening and locking, allocating memory in a cache,
+ initializing a process for reading records, reading records in a
+ loop until the thread is killed or there are no more to read,
+ storing a modified record for the table, and — after the
+ loop is through — possibly writing to the log.
+ Incidentally, a transactional table is usually a BDB or an
+ InnoDB table.
+ </para>
- <para>
- Obviously we've picked out what's easy to follow, and we're not
- pretending it's all smooth sailing. But this is actual code and
- you can check it out yourself.
- </para>
+ <para>
+ Obviously we've picked out what's easy to follow, and we're not
+ pretending it's all smooth sailing. But this is actual code and
+ you can check it out yourself.
+ </para>
- <para>
- <emphasis role="bold">The Skeleton Of The Server Code</emphasis>
- </para>
+ </section>
- <para>
- And now we're going to walk through something harder, namely the
- server.
- </para>
+ <section id="guided-tour-skeleton">
- <para>
- WARNING WARNING WARNING: code changes constantly, so names and
- parameters may have changed by the time you read this.
- </para>
+ <title>The Skeleton Of The Server Code</title>
- <para>
- Important programs we'll be walking through:
- </para>
+ <para>
+ And now we're going to walk through something harder, namely the
+ server.
+ </para>
+ <para>
+ WARNING WARNING WARNING: code changes constantly, so names and
+ parameters may have changed by the time you read this.
+ </para>
+
+ <para>
+ Important programs we'll be walking through:
+ </para>
+
<programlisting>
/sql/mysqld.cc
/sql/sql_parse.cc
@@ -1189,31 +1276,31 @@
/myisam/mi_write.c
</programlisting>
- <para>
- This is not as simple as what we've just done. In fact we'll need
- multiple pages to walk through this one, and that's despite our
- use of truncation and condensation again. But the server is
- important, and if you can grasp what we're doing with it, you'll
- have grasped the essence of what the MySQL source code is all
- about.
- </para>
+ <para>
+ This is not as simple as what we've just done. In fact we'll
+ need multiple pages to walk through this one, and that's despite
+ our use of truncation and condensation again. But the server is
+ important, and if you can grasp what we're doing with it, you'll
+ have grasped the essence of what the MySQL source code is all
+ about.
+ </para>
- <para>
- We'll mostly be looking at programs in the sql directory, which is
- where mysqld and most of the programs for the SQL engine code are
- stored.
- </para>
+ <para>
+ We'll mostly be looking at programs in the sql directory, which
+ is where mysqld and most of the programs for the SQL engine code
+ are stored.
+ </para>
- <para>
- Our objective is to follow the server from the time it starts up,
- through a single INSERT statement that it receives from a client,
- to the point where it finally performs the low level write in the
- MyISAM file.
- </para>
+ <para>
+ Our objective is to follow the server from the time it starts
+ up, through a single INSERT statement that it receives from a
+ client, to the point where it finally performs the low level
+ write in the MyISAM file.
+ </para>
- <para>
- Walking Through The Server Code: /sql/mysqld.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/mysqld.cc
+ </para>
<programlisting>
int main(int argc, char **argv)
@@ -1234,48 +1321,49 @@
}
</programlisting>
- <para>
- Here is where it all starts, in the main function of mysqld.cc.
- </para>
+ <para>
+ Here is where it all starts, in the main function of mysqld.cc.
+ </para>
- <para>
- Notice that we show a directory name and program name just above
- this snippet. We will do the same for all the snippets in this
- series.
- </para>
+ <para>
+ Notice that we show a directory name and program name just above
+ this snippet. We will do the same for all the snippets in this
+ series.
+ </para>
- <para>
- By glancing at this snippet for a few seconds, you will probably
- see that the main function is doing some initial checks on
- startup, is initializing some components, is calling a function
- named handle_connections_sockets, and then is exiting. It's
- possible that acl stands for "access control" and it's interesting
- that DBUG_PRINT is something from Fred Fish's debug library, which
- we've mentioned before. But we must not digress.
- </para>
+ <para>
+ By glancing at this snippet for a few seconds, you will probably
+ see that the main function is doing some initial checks on
+ startup, is initializing some components, is calling a function
+ named handle_connections_sockets, and then is exiting. It's
+ possible that acl stands for "access control" and it's
+ interesting that DBUG_PRINT is something from Fred Fish's debug
+ library, which we've mentioned before. But we must not digress.
+ </para>
- <para>
- In fact there are 150 code lines in the main function, and we're
- only showing 13 code lines. That will give you an idea of how much
- we are shaving and pruning. We threw away the error checks, the
- side paths, the optional code, and the variables. But we did not
- change what was left. You will be able to find these lines if you
- take an editor to the mysqld.cc program, and the same applies for
- all the other routines in the snippets in this series.
- </para>
+ <para>
+ In fact there are 150 code lines in the main function, and we're
+ only showing 13 code lines. That will give you an idea of how
+ much we are shaving and pruning. We threw away the error checks,
+ the side paths, the optional code, and the variables. But we did
+ not change what was left. You will be able to find these lines
+ if you take an editor to the mysqld.cc program, and the same
+ applies for all the other routines in the snippets in this
+ series.
+ </para>
- <para>
- The one thing you won't see in the actual source code is the
- little marker "// !". This marker will always be on the line of
- the function that will be the subject of the next snippet. In this
- case, it means that the next snippet will show the
- handle_connection_sockets function. To prove that, let's go to the
- next snippet.
- </para>
+ <para>
+ The one thing you won't see in the actual source code is the
+ little marker "// !". This marker will always be on the line of
+ the function that will be the subject of the next snippet. In
+ this case, it means that the next snippet will show the
+ handle_connection_sockets function. To prove that, let's go to
+ the next snippet.
+ </para>
- <para>
- Walking Through The Server Code: /sql/mysqld.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/mysqld.cc
+ </para>
<programlisting>
handle_connections_sockets (arg __attribute__((unused))
@@ -1296,29 +1384,29 @@
}
</programlisting>
- <para>
- Inside handle_connections_sockets you'll see the hallmarks of a
- classic client/server architecture. In a classic client/server,
- the server has a main thread which is always listening for
- incoming requests from new clients. Once it receives such a
- request, it assigns resources which will be exclusive to that
- client. In particular, the main thread will spawn a new thread
- just to handle the connection. Then the main server will loop and
- listen for new connections — but we will leave it and follow
- the new thread.
- </para>
+ <para>
+ Inside handle_connections_sockets you'll see the hallmarks of a
+ classic client/server architecture. In a classic client/server,
+ the server has a main thread which is always listening for
+ incoming requests from new clients. Once it receives such a
+ request, it assigns resources which will be exclusive to that
+ client. In particular, the main thread will spawn a new thread
+ just to handle the connection. Then the main server will loop
+ and listen for new connections — but we will leave it and
+ follow the new thread.
+ </para>
- <para>
- As well as the sockets code that we chose to display here, there
- are several variants of this thread loop, because clients can
- choose to connect in other ways, for example with named pipes or
- with shared memory. But the important item to note from this
- section is that the server is spawning new threads.
- </para>
+ <para>
+ As well as the sockets code that we chose to display here, there
+ are several variants of this thread loop, because clients can
+ choose to connect in other ways, for example with named pipes or
+ with shared memory. But the important item to note from this
+ section is that the server is spawning new threads.
+ </para>
- <para>
- Walking Through The Server Code: /sql/mysqld.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/mysqld.cc
+ </para>
<programlisting>
create_new_thread(THD *thd)
@@ -1331,17 +1419,17 @@
}
</programlisting>
- <para>
- Here is a close look at the routine that spawns the new thread.
- The noticeable detail is that, as you can see, it uses a mutex or
- mutual exclusion object. MySQL has a great variety of mutexes that
- it uses to keep actions of all the threads from conflicting with
- each other.
- </para>
+ <para>
+ Here is a close look at the routine that spawns the new thread.
+ The noticeable detail is that, as you can see, it uses a mutex
+ or mutual exclusion object. MySQL has a great variety of mutexes
+ that it uses to keep actions of all the threads from conflicting
+ with each other.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
handle_one_connection(THD *thd)
@@ -1357,22 +1445,22 @@
packet=(char*) net->read_pos;
</programlisting>
- <para>
- With this snippet, we've wandered out of mysqld.cc. Now, we're in
- the sql_parse program, still in the sql directory. This is where
- the session's big loop is.
- </para>
+ <para>
+ With this snippet, we've wandered out of mysqld.cc. Now, we're
+ in the sql_parse program, still in the sql directory. This is
+ where the session's big loop is.
+ </para>
- <para>
- The loop repeatedly gets and does commands. When it ends, the
- connection closes. At that point, the thread will end and the
- resources for it will be deallocated.
- </para>
+ <para>
+ The loop repeatedly gets and does commands. When it ends, the
+ connection closes. At that point, the thread will end and the
+ resources for it will be deallocated.
+ </para>
- <para>
- But we're more interested in what happens inside the loop, when we
- call the do_command function.
- </para>
+ <para>
+ But we're more interested in what happens inside the loop, when
+ we call the do_command function.
+ </para>
<programlisting>
Graphic:
@@ -1384,18 +1472,18 @@
INSERT INTO Table1 VALUES (1);
</programlisting>
- <para>
- To put it graphically, at this point there is a long-lasting
- connection between the client and one server thread. Message
- packets will go back and forth between them through this
- connection. For today's tour, let's assume that the client passes
- the INSERT statement shown on the Graphic, for the server to
- process.
- </para>
+ <para>
+ To put it graphically, at this point there is a long-lasting
+ connection between the client and one server thread. Message
+ packets will go back and forth between them through this
+ connection. For today's tour, let's assume that the client
+ passes the INSERT statement shown on the Graphic, for the server
+ to process.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
bool do_command(THD *thd)
@@ -1409,34 +1497,34 @@
}
</programlisting>
- <para>
- You've probably noticed by now that whenever we call a lower-level
- routine, we pass an argument named thd, which is an abbreviation
- for the word thread (we think). This is the essential context
- which we must never lose.
- </para>
+ <para>
+ You've probably noticed by now that whenever we call a
+ lower-level routine, we pass an argument named thd, which is an
+ abbreviation for the word thread (we think). This is the
+ essential context which we must never lose.
+ </para>
- <para>
- The my_net_read function is in another program called net_serv.cc.
- It gets a packet from the client, uncompresses it, and strips the
- header.
- </para>
+ <para>
+ The my_net_read function is in another program called
+ net_serv.cc. It gets a packet from the client, uncompresses it,
+ and strips the header.
+ </para>
- <para>
- Once that's done, we've got a multi-byte variable named packet
- which contains what the client has sent. The first byte is
- important because it contains a code identifying the type of
- message.
- </para>
+ <para>
+ Once that's done, we've got a multi-byte variable named packet
+ which contains what the client has sent. The first byte is
+ important because it contains a code identifying the type of
+ message.
+ </para>
- <para>
- We'll pass that and the rest of the packet on to the
- dispatch_command function.
- </para>
+ <para>
+ We'll pass that and the rest of the packet on to the
+ dispatch_command function.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
bool dispatch_command(enum enum_server_command command, THD *thd,
@@ -1459,37 +1547,38 @@
}
</programlisting>
- <para>
- And here's just part of a very large switch statement in
- sql_parse.cc. The snippet doesn't have room to show the rest, but
- you'll see when you look at the dispatch_command function that
- there are more case statements after the ones that you see here.
- </para>
+ <para>
+ And here's just part of a very large switch statement in
+ sql_parse.cc. The snippet doesn't have room to show the rest,
+ but you'll see when you look at the dispatch_command function
+ that there are more case statements after the ones that you see
+ here.
+ </para>
- <para>
- There will be — we're going into list mode now and just
- reciting the rest of the items in the switch statement —
- code for prepare, close statement, query, quit, create database,
- drop database, dump binary log, refresh, statistics, get process
- info, kill process, sleep, connect, and several minor commands.
- This is the big junction.
- </para>
+ <para>
+ There will be — we're going into list mode now and just
+ reciting the rest of the items in the switch statement —
+ code for prepare, close statement, query, quit, create database,
+ drop database, dump binary log, refresh, statistics, get process
+ info, kill process, sleep, connect, and several minor commands.
+ This is the big junction.
+ </para>
- <para>
- We have cut out the code for all of the cases except for two,
- COM_EXECUTE and COM_PREPARE.
- </para>
+ <para>
+ We have cut out the code for all of the cases except for two,
+ COM_EXECUTE and COM_PREPARE.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_prepare.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_prepare.cc
+ </para>
- <para>
- We are not going to follow what happens with COM_PREPARE. Instead,
- we are going to follow the code after COM_EXECUTE. But we'll have
- to digress from our main line for a moment and explain what the
- prepare does.
- </para>
+ <para>
+ We are not going to follow what happens with COM_PREPARE.
+ Instead, we are going to follow the code after COM_EXECUTE. But
+ we'll have to digress from our main line for a moment and
+ explain what the prepare does.
+ </para>
<programlisting>
"Prepare:
@@ -1499,26 +1588,26 @@
metadata information (if any)"
</programlisting>
- <para>
- The prepare is the step that must happen before execute happens.
- It consists of checking for syntax errors, looking up any tables
- and columns referenced in the statement, and setting up tables for
- the execute to use. Once a prepare is done, an execute can be done
- multiple times without having to go through the syntax checking
- and table lookups again.
- </para>
+ <para>
+ The prepare is the step that must happen before execute happens.
+ It consists of checking for syntax errors, looking up any tables
+ and columns referenced in the statement, and setting up tables
+ for the execute to use. Once a prepare is done, an execute can
+ be done multiple times without having to go through the syntax
+ checking and table lookups again.
+ </para>
- <para>
- Since we're not going to walk through the COM_PREPARE code, we
- decided not to show its code at this point. Instead, we have cut
- and pasted some code comments that describe prepare. All we're
- illustrating here is that there are comments in the code, so you
- will have aid when you look harder at the prepare code.
- </para>
+ <para>
+ Since we're not going to walk through the COM_PREPARE code, we
+ decided not to show its code at this point. Instead, we have cut
+ and pasted some code comments that describe prepare. All we're
+ illustrating here is that there are comments in the code, so you
+ will have aid when you look harder at the prepare code.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
bool dispatch_command(enum enum_server_command command, THD *thd,
@@ -1541,16 +1630,16 @@
}
</programlisting>
- <para>
- Let's return to the grand central junction again in sql_parse.cc
- for a moment. The thing to note on this snippet is that the line
- which we're really going to follow is what happens for
- COM_EXECUTE.
- </para>
+ <para>
+ Let's return to the grand central junction again in sql_parse.cc
+ for a moment. The thing to note on this snippet is that the line
+ which we're really going to follow is what happens for
+ COM_EXECUTE.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_prepare.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_prepare.cc
+ </para>
<programlisting>
void mysql_stmt_execute(THD *thd, char *packet)
@@ -1565,25 +1654,25 @@
}
</programlisting>
- <para>
- In this case, the line that we're following is the line that
- executes a statement.
- </para>
+ <para>
+ In this case, the line that we're following is the line that
+ executes a statement.
+ </para>
- <para>
- Notice how we keep carrying the THD thread and the packet along
- with us, and notice that we expect to find a prepared statement
- waiting for us, since this is the execute phase. Notice as well
- that we continue to sprinkle error-related functions that begin
- with the letters DBUG, for use by the debug library. Finally,
- notice that the identifier "stmt" is the same name that ODBC uses
- for the equivalent object. We try to use standard names when they
- fit.
- </para>
+ <para>
+ Notice how we keep carrying the THD thread and the packet along
+ with us, and notice that we expect to find a prepared statement
+ waiting for us, since this is the execute phase. Notice as well
+ that we continue to sprinkle error-related functions that begin
+ with the letters DBUG, for use by the debug library. Finally,
+ notice that the identifier "stmt" is the same name that ODBC
+ uses for the equivalent object. We try to use standard names
+ when they fit.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
void mysql_execute_command(THD *thd)
@@ -1598,15 +1687,15 @@
}
</programlisting>
- <para>
- In the mysql_execute_command function. we encounter another
- junction. One of the items in the switch statement is named
- SQLCOM_INSERT.
- </para>
+ <para>
+ In the mysql_execute_command function. we encounter another
+ junction. One of the items in the switch statement is named
+ SQLCOM_INSERT.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_parse.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_parse.cc
+ </para>
<programlisting>
case SQLCOM_INSERT:
@@ -1633,24 +1722,24 @@
}
</programlisting>
- <para>
- For this snippet, we've blown up the code around the SQLCOM_INSERT
- case in the mysql_execute_command function. The first thing to do
- is check whether the user has the appropriate privileges for doing
- an INSERT into the table, and this is the place where the server
- checks for that, by calling the check_access and check_grant
- functions. It would be tempting to follow those functions, but
- those are side paths. Instead, we'll follow the path where the
- work is going on.
- </para>
+ <para>
+ For this snippet, we've blown up the code around the
+ SQLCOM_INSERT case in the mysql_execute_command function. The
+ first thing to do is check whether the user has the appropriate
+ privileges for doing an INSERT into the table, and this is the
+ place where the server checks for that, by calling the
+ check_access and check_grant functions. It would be tempting to
+ follow those functions, but those are side paths. Instead, we'll
+ follow the path where the work is going on.
+ </para>
- <para>
- Walking Through The Server Code: Navigation Aid
- </para>
+ <para>
+ Walking Through The Server Code: Navigation Aid
+ </para>
- <para>
- Some program names in the /sql directory:
- </para>
+ <para>
+ Some program names in the /sql directory:
+ </para>
<programlisting>
Program Name SQL statement type
@@ -1667,35 +1756,36 @@
sql_update.cc UPDATE
</programlisting>
- <para>
- Question: Where will mysql_insert() be?
- </para>
+ <para>
+ Question: Where will mysql_insert() be?
+ </para>
- <para>
- The line that we're following will take us next to a routine named
- mysql_insert. Sometimes it's difficult to guess what program a
- routine will be in, because MySQL has no consistent naming
- convention. However, here is one aid to navigation that works for
- some statement types. In the sql directory, the names of some
- programs correspond to statement types. This happens to be the
- case for INSERT, for instance. So the mysql_insert program will be
- in the program sql_insert.cc. But there's no reliable rule.
- </para>
+ <para>
+ The line that we're following will take us next to a routine
+ named mysql_insert. Sometimes it's difficult to guess what
+ program a routine will be in, because MySQL has no consistent
+ naming convention. However, here is one aid to navigation that
+ works for some statement types. In the sql directory, the names
+ of some programs correspond to statement types. This happens to
+ be the case for INSERT, for instance. So the mysql_insert
+ program will be in the program sql_insert.cc. But there's no
+ reliable rule.
+ </para>
- <para>
- (Let's add here a few sentences about the tags 'ctags' program.
- When an editor supports ctags (and the list is long, but vi and
- emacs of course are there), the function definition is one key
- press away - no guessing involved. In the above case, a vim user
- could press ^] on mysql_insert name and vim would open
- sql_insert.cc and position the curson on the first line of the
- mysql_insert() function. The tags help can be indispensable in
- everyday work.)
- </para>
+ <para>
+ (Let's add here a few sentences about the tags 'ctags' program.
+ When an editor supports ctags (and the list is long, but vi and
+ emacs of course are there), the function definition is one key
+ press away - no guessing involved. In the above case, a vim user
+ could press ^] on mysql_insert name and vim would open
+ sql_insert.cc and position the curson on the first line of the
+ mysql_insert() function. The tags help can be indispensable in
+ everyday work.)
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_insert.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_insert.cc
+ </para>
<programlisting>
int mysql_insert(THD *thd,TABLE_LIST *table_list, List<Item> &fields,
@@ -1716,30 +1806,31 @@
}
</programlisting>
- <para>
- For the mysql_insert routine, we're just going to read what's in
- the snippet. What we're trying to do here is highlight the fact
- that the function names and variable names are nearly English.
- </para>
+ <para>
+ For the mysql_insert routine, we're just going to read what's in
+ the snippet. What we're trying to do here is highlight the fact
+ that the function names and variable names are nearly English.
+ </para>
- <para>
- Okay, we start by opening a table. Then, if a check of the fields
- in the INSERT fails, or if an attempt to set up the tables fails,
- or if an attempt to set up the fields fails, we'll abort.
- </para>
+ <para>
+ Okay, we start by opening a table. Then, if a check of the
+ fields in the INSERT fails, or if an attempt to set up the
+ tables fails, or if an attempt to set up the fields fails, we'll
+ abort.
+ </para>
- <para>
- Next, we'll fill the record buffer with values. Then we'll write
- the record. Then we'll invalidate the query cache. Remember, by
- the way, that MySQL stores frequently-used select statements and
- result sets in memory as an optimization, but once the insert
- succeeds the stored sets are invalid. Finally, we'll unlock the
- tables.
- </para>
+ <para>
+ Next, we'll fill the record buffer with values. Then we'll write
+ the record. Then we'll invalidate the query cache. Remember, by
+ the way, that MySQL stores frequently-used select statements and
+ result sets in memory as an optimization, but once the insert
+ succeeds the stored sets are invalid. Finally, we'll unlock the
+ tables.
+ </para>
- <para>
- Walking Through The Server Code: /sql/sql_insert.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/sql_insert.cc
+ </para>
<programlisting>
int write_record(TABLE *table,COPY_INFO *info)
@@ -1748,18 +1839,18 @@
}
</programlisting>
- <para>
- You can see from our marker that we're going to follow the line
- that contains the words 'write row'. But this is not an ordinary
- function call, so people who are just reading the code without the
- aid of a debugger can easily miss what the next point is in the
- line of execution here. The fact is, 'write_row' can take us to
- one of several different places.
- </para>
+ <para>
+ You can see from our marker that we're going to follow the line
+ that contains the words 'write row'. But this is not an ordinary
+ function call, so people who are just reading the code without
+ the aid of a debugger can easily miss what the next point is in
+ the line of execution here. The fact is, 'write_row' can take us
+ to one of several different places.
+ </para>
- <para>
- Walking Through The Server Code: /sql/handler.h
- </para>
+ <para>
+ Walking Through The Server Code: /sql/handler.h
+ </para>
<programlisting>
/* The handler for a table type.
@@ -1780,24 +1871,24 @@
virtual int write_row(byte * buf)=0;
</programlisting>
- <para>
- To see what the write_row statement is doing, we'll have to look
- at one of the include files. In handler.h on the sql directory, we
- find that write_row is associated with a handler for a table. This
- definition is telling us that the address in write_row will vary
- — it gets filled in at run time. In fact, there are several
- possible addresses.
- </para>
+ <para>
+ To see what the write_row statement is doing, we'll have to look
+ at one of the include files. In handler.h on the sql directory,
+ we find that write_row is associated with a handler for a table.
+ This definition is telling us that the address in write_row will
+ vary — it gets filled in at run time. In fact, there are
+ several possible addresses.
+ </para>
- <para>
- There is one address for each handler. In our case, since we're
- using the default values, the value at this point will be the
- address of write_row in the MyISAM handler program.
- </para>
+ <para>
+ There is one address for each handler. In our case, since we're
+ using the default values, the value at this point will be the
+ address of write_row in the MyISAM handler program.
+ </para>
- <para>
- Walking Through The Server Code: /sql/ha_myisam.cc
- </para>
+ <para>
+ Walking Through The Server Code: /sql/ha_myisam.cc
+ </para>
<programlisting>
int ha_myisam::write_row(byte * buf)
@@ -1816,17 +1907,18 @@
}
</programlisting>
- <para>
- And that brings us to write_row in the ha_myisam.cc program.
- Remember we told you that these programs beginning with the
- letters ha are interfaces to handlers, and this one is the
- interface to the myisam handler. We have at last reached the point
- where we're ready to call something in the handler package.
- </para>
+ <para>
+ And that brings us to write_row in the ha_myisam.cc program.
+ Remember we told you that these programs beginning with the
+ letters ha are interfaces to handlers, and this one is the
+ interface to the myisam handler. We have at last reached the
+ point where we're ready to call something in the handler
+ package.
+ </para>
- <para>
- Walking Through The Server Code: /myisam/mi_write.c
- </para>
+ <para>
+ Walking Through The Server Code: /myisam/mi_write.c
+ </para>
<programlisting>
int mi_write(MI_INFO *info, byte *record)
@@ -1844,22 +1936,22 @@
... to be continued in next snippet
</programlisting>
- <para>
- Notice that at this point there is no more referencing of tables,
- the comments are about files and index keys. We have reached the
- bottom level at last. Notice as well that we are now in a C
- program, not a C++ program.
- </para>
+ <para>
+ Notice that at this point there is no more referencing of
+ tables, the comments are about files and index keys. We have
+ reached the bottom level at last. Notice as well that we are now
+ in a C program, not a C++ program.
+ </para>
- <para>
- In this first half of the mi_write function, we see a call which
- is clearly commented. This is where checking happens for
- uniqueness (not the UNIQUE constraint, but an internal matter).
- </para>
+ <para>
+ In this first half of the mi_write function, we see a call which
+ is clearly commented. This is where checking happens for
+ uniqueness (not the UNIQUE constraint, but an internal matter).
+ </para>
- <para>
- Walking Through The Server Code: /myisam/mi_write.c
- </para>
+ <para>
+ Walking Through The Server Code: /myisam/mi_write.c
+ </para>
<programlisting>
... continued from previous snippet
@@ -1876,23 +1968,23 @@
}
</programlisting>
- <para>
- In this second half of the mi_write function, we see another clear
- comment, to the effect that this is where the new keys are made
- for any indexed columns. Then we see the culmination of all that
- the last 20 snippets have been preparing, the moment we've all
- been waiting for, the writing of the record.
- </para>
+ <para>
+ In this second half of the mi_write function, we see another
+ clear comment, to the effect that this is where the new keys are
+ made for any indexed columns. Then we see the culmination of all
+ that the last 20 snippets have been preparing, the moment we've
+ all been waiting for, the writing of the record.
+ </para>
- <para>
- And, since the object of the INSERT statement is ultimately to
- cause a write to a record in a file, that's that. The server has
- done the job.
- </para>
+ <para>
+ And, since the object of the INSERT statement is ultimately to
+ cause a write to a record in a file, that's that. The server has
+ done the job.
+ </para>
- <para>
- Walking Through The Server Code: Stack Trace
- </para>
+ <para>
+ Walking Through The Server Code: Stack Trace
+ </para>
<programlisting>
main in /sql/mysqld.cc
@@ -1909,30 +2001,30 @@
mi_write in /myisam/mi_write.c
</programlisting>
- <para>
- And now here's a look at what's above us on the stack, or at least
- an idea of how we got here. We started with the main program in
- mysqld.cc. We proceeded through the creation of a thread for the
- client, the several junction processes that determined where we're
- heading, the parsing and initial execution of an SQL statement,
- the decision to invoke the MyISAM handler, and the writing of the
- row. We ended in a low level place, where we're calling the
- routines that write to the file. That's about as low as we should
- go today.
- </para>
+ <para>
+ And now here's a look at what's above us on the stack, or at
+ least an idea of how we got here. We started with the main
+ program in mysqld.cc. We proceeded through the creation of a
+ thread for the client, the several junction processes that
+ determined where we're heading, the parsing and initial
+ execution of an SQL statement, the decision to invoke the MyISAM
+ handler, and the writing of the row. We ended in a low level
+ place, where we're calling the routines that write to the file.
+ That's about as low as we should go today.
+ </para>
- <para>
- The server program would, of course, continue by returning several
- times in a row, sending a packet to the client saying "Okay", and
- ending up back in the loop inside the handle_one_connection
- function.
- </para>
+ <para>
+ The server program would, of course, continue by returning
+ several times in a row, sending a packet to the client saying
+ "Okay", and ending up back in the loop inside the
+ handle_one_connection function.
+ </para>
- <para>
- We, instead, will pause for a moment in awe at the amount of code
- we've just flitted past. And that will end our walk through the
- server code.
- </para>
+ <para>
+ We, instead, will pause for a moment in awe at the amount of
+ code we've just flitted past. And that will end our walk through
+ the server code.
+ </para>
<programlisting>
Graphic: A Chunk of MyISAM File
@@ -1949,39 +2041,40 @@
F1 61 62 63 00 F5 64 00 66 00 ... .abc..d e.
</programlisting>
- <para>
- Continuing with our worm's-eye view, let's glance at the structure
- of a record in a MyISAM file.
- </para>
+ <para>
+ Continuing with our worm's-eye view, let's glance at the
+ structure of a record in a MyISAM file.
+ </para>
- <para>
- The SQL statements on this graphic show a table definition and
- some insert statements that we used to populate the table.
- </para>
+ <para>
+ The SQL statements on this graphic show a table definition and
+ some insert statements that we used to populate the table.
+ </para>
- <para>
- The final line on the graphic is a hexadecimal dump display of the
- two records that we ended up with, as taken from the MyISAM file
- for Table1.
- </para>
+ <para>
+ The final line on the graphic is a hexadecimal dump display of
+ the two records that we ended up with, as taken from the MyISAM
+ file for Table1.
+ </para>
- <para>
- The thing to notice here is that the records are stored compactly.
- There is one byte at the start of each record — F1 for the
- first record and F5 for the second record — which contains a
- bit list.
- </para>
+ <para>
+ The thing to notice here is that the records are stored
+ compactly. There is one byte at the start of each record —
+ F1 for the first record and F5 for the second record —
+ which contains a bit list.
+ </para>
- <para>
- When a bit is on, that means its corresponding field is NULL.
- That's why the second row, which has a NULL in the second column,
- or field, has a different header byte from the first row.
- </para>
+ <para>
+ When a bit is on, that means its corresponding field is NULL.
+ That's why the second row, which has a NULL in the second
+ column, or field, has a different header byte from the first
+ row.
+ </para>
- <para>
- Complications are possible, but a simple record really does look
- this simple.
- </para>
+ <para>
+ Complications are possible, but a simple record really does look
+ this simple.
+ </para>
<programlisting>
Graphic: A Chunk of InnoDB File
@@ -1996,40 +2089,40 @@
50 50 Field3 'PP'
</programlisting>
- <para>
- If, on the other hand, you look at an InnoDB file, you'll find
- that it's got more complexities in the storage. The details are
- elsewhere in this document. But here's an introductory look.
- </para>
+ <para>
+ If, on the other hand, you look at an InnoDB file, you'll find
+ that it's got more complexities in the storage. The details are
+ elsewhere in this document. But here's an introductory look.
+ </para>
- <para>
- The header here begins with offsets — unlike MyISAM, which
- has no offsets. So you'd have to go through column 1 before
- getting to column 2.
- </para>
+ <para>
+ The header here begins with offsets — unlike MyISAM, which
+ has no offsets. So you'd have to go through column 1 before
+ getting to column 2.
+ </para>
- <para>
- Then there is a fixed header — the extra bytes.
- </para>
+ <para>
+ Then there is a fixed header — the extra bytes.
+ </para>
- <para>
- Then comes the record proper. The first fields of a typical record
- contain information that the user won't see, such as a row ID, a
- transaction ID, and a rollback pointer. This part would look
- different if the user had defined a primary key during the CREATE
- TABLE statement.
- </para>
+ <para>
+ Then comes the record proper. The first fields of a typical
+ record contain information that the user won't see, such as a
+ row ID, a transaction ID, and a rollback pointer. This part
+ would look different if the user had defined a primary key
+ during the CREATE TABLE statement.
+ </para>
- <para>
- And finally there are the column contents — the string of Ps
- at the end of the snippet here. You can see that InnoDB does more
- administrating.
- </para>
+ <para>
+ And finally there are the column contents — the string of
+ Ps at the end of the snippet here. You can see that InnoDB does
+ more administrating.
+ </para>
- <para>
- There's been a recent change for InnoDB; what you see above is
- from a database made before version 5.0.
- </para>
+ <para>
+ There's been a recent change for InnoDB; what you see above is
+ from a database made before version 5.0.
+ </para>
<programlisting>
Graphic: A Packet
@@ -2042,68 +2135,72 @@
Message Content
</programlisting>
- <para>
- Our final worm's-eye look at a physical structure will be a look
- at packets.
- </para>
+ <para>
+ Our final worm's-eye look at a physical structure will be a look
+ at packets.
+ </para>
- <para>
- By packet, we mean: what's the format of a message that the client
- sends over the tcp/ip line to the server — and what does the
- server send back?
- </para>
+ <para>
+ By packet, we mean: what's the format of a message that the
+ client sends over the tcp/ip line to the server — and what
+ does the server send back?
+ </para>
- <para>
- Here we're not displaying a dump. If you want to see hexadecimal
- dumps of the contents of packets, this document is full of them.
- We're just going to note that a typical message will have a
- header, an identifier, and a length, followed by the message
- contents.
- </para>
+ <para>
+ Here we're not displaying a dump. If you want to see hexadecimal
+ dumps of the contents of packets, this document is full of them.
+ We're just going to note that a typical message will have a
+ header, an identifier, and a length, followed by the message
+ contents.
+ </para>
- <para>
- Admittedly this isn't following a standard like ISO's RDA or IBM's
- DRDA, but it's documented so if you want to go out and write your
- own type 4 JDBC driver, you've got what you need here. (Subject to
- license restrictions, of course.) But a word of advice on that
- last point: it's already been done. Mark Matthews wrote it
- originally, it's all in "MySQL Connector/J".
- </para>
+ <para>
+ Admittedly this isn't following a standard like ISO's RDA or
+ IBM's DRDA, but it's documented so if you want to go out and
+ write your own type 4 JDBC driver, you've got what you need
+ here. (Subject to license restrictions, of course.) But a word
+ of advice on that last point: it's already been done. Mark
+ Matthews wrote it originally, it's all in "MySQL Connector/J".
+ </para>
- <para>
- <emphasis role="bold">The Last Subheader</emphasis>
- </para>
+ </section>
- <para>
- Okay, let's back up and restate. In this walkthrough, we've told
- you four main things.
- </para>
+ <section id="guided-tour-recap">
- <para>
- One: How to get the MySQL source.
- </para>
+ <title>Recap</title>
- <para>
- Two: What's in each directory in the source.
- </para>
+ <para>
+ Okay, let's back up and restate. In this walkthrough, we've told
+ you four main things.
+ </para>
- <para>
- Three: The main sequence, as one walks through the server code.
- </para>
+ <para>
+ One: How to get the MySQL source.
+ </para>
- <para>
- Four: What physical structures look like.
- </para>
+ <para>
+ Two: What's in each directory in the source.
+ </para>
- <para>
- We worked hard to make a description of the MySQL source that is
- simple, without distorting. If you were able to follow all that
- we've said, then that's wonderful, congratulations. If you ended
- up thinking that MySQL is really simple, well that's not what we
- wanted to convey, but we think you'll be disabused of that notion
- when you have a look at the code yourself.
- </para>
+ <para>
+ Three: The main sequence, as one walks through the server code.
+ </para>
+ <para>
+ Four: What physical structures look like.
+ </para>
+
+ <para>
+ We worked hard to make a description of the MySQL source that is
+ simple, without distorting. If you were able to follow all that
+ we've said, then that's wonderful, congratulations. If you ended
+ up thinking that MySQL is really simple, well that's not what we
+ wanted to convey, but we think you'll be disabused of that
+ notion when you have a look at the code yourself.
+ </para>
+
+ </section>
+
</chapter>
<chapter id="coding-guidelines">
@@ -10479,6 +10576,605 @@
</chapter>
+ <chapter id="prepared-stored">
+
+ <title>Prepared Statements and Stored Routines</title>
+
+ <para>
+ Let us start with a general description of the MySQL statement
+ processing workflow in order to provide the reader with
+ understanding of the problem of reexecution and vocabulary for the
+ following sections.
+ </para>
+
+ <para>
+ Conventional statements, that is, SQL queries sent in
+ <literal>COM_QUERY</literal> protocol packet, are the only
+ statements present in MySQL server prior to version 4.1. Execution
+ of such statements is performed in a batch mode, one query
+ processed by the server at a time. The original implementation is
+ streamlined for this mode and has a single global connection state
+ <literal>THD</literal> shared among all operational steps.
+ </para>
+
+ <para>
+ When executing a query in conventional mode, the server
+ sequentially parses its text, acquires table level locks, analyzes
+ the parsed tree, builds an execution plan, executes the built plan
+ and releases the locks.
+ </para>
+
+ <para>
+ Memory for parsing is allocated using block allocator
+ <literal>MEM_ROOT</literal> in 4k chunks and freed once in the end
+ of execution. Memory for execution is allocated in the memory root
+ of the parsed tree, as well as in the system heap, and in some
+ cases in local "memory roots" of execution modules.
+ </para>
+
+ <para>
+ The role of the parser is to create a set of objects to represent
+ the query. E.g. for a <literal>SELECT</literal> statement, this
+ set includes a list of Item's for <literal>SELECT</literal> list,
+ a list of tables (<literal>TABLE_LIST</literal> object for each
+ table) for <literal>FROM</literal> clause, and a tree of Item's
+ for <literal>WHERE</literal> clause.
+ </para>
+
+ <para>
+ During context analysis phase, links are established from the
+ parsed tree to the physical objects of the database, such as open
+ tables and table columns. A physical table is represented by a
+ heir of class handler that corresponds to the storage engine the
+ table belongs to, and is saved in
+ <literal>TABLE_LIST::file</literal>.
+ </para>
+
+ <para>
+ When context analysis is done, the query optimizer is run. It
+ performs two major tasks:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Query transformation — a transformation of the parsed
+ tree to an equivalent one, which is simpler and more efficient
+ to execute.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Creation of an execution plan, including evaluation of an
+ order of joins and initialization of methods to access the
+ used tables. At this step parts of the execution plan are
+ attached to the parsed tree.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ Finally, the query is passed to the execution runtime — an
+ interpreter that operates with and modifies both the parsed tree
+ and the execution plan in order to execute the query.
+ </para>
+
+ <para>
+ It should be noted that the overall procedure is infamous for
+ breaking borders between abstraction layers. For example, MySQL
+ performs [sub]query transformation during context analysis;
+ moreover, most parts of the code rely on the fact that
+ <literal>THD</literal> is processing only one statement at a time.
+ </para>
+
+ <section id="prepared-stored-reexecution">
+
+ <title>Statement Re-execution Requirements</title>
+
+ <para>
+ Features of MySQL 4.1 and 5.0 put a new demand on the execution
+ process: prepared statements and stored routines need to reuse
+ the same parsed tree to execute a query many times.
+ </para>
+
+ <para>
+ So far no easy mechanism that would allow query reexecution
+ using the conventional query processing code has been found. For
+ instance, copying of the parsed tree before each reexecution is
+ not simple to implement as a parsed tree, which can contain
+ instances of more than 300 different classes, has a lot of
+ cross-references between its objects.
+ </para>
+
+ <para>
+ The present solution introduces a concept of change management
+ for the changes of the parsed tree and is largely a unification
+ of numerous fixes of bugs in reexecution. The solution has two
+ aspects.
+ </para>
+
+ <para>
+ The first one is that modifications of the parsed tree are
+ tracked and a way to restore the tree to a state that allows
+ reexecution is introduced.
+ </para>
+
+ <para>
+ The second aspect is that a dedicated block allocator (memory
+ root) is used to store the parsed tree, and the memory allocated
+ in this memory root is freed only when the parsed tree is
+ destroyed. Later this memory root will be denoted as the
+ permanent memory root of a statement.
+ </para>
+
+ <para>
+ In order to properly restore the parsed tree to a usable state,
+ all modifications of the tree are classified as destructive or
+ non-destructive and an appropriate action is taken for every
+ type of modification.
+ </para>
+
+ <para>
+ A non-destructive modification does not depend on actual values
+ of prepared statement placeholders or contents of the tables
+ used in a query. Such modification is [and should be, for future
+ changes] made only once and the memory for it is allocated in
+ the permanent memory root of the statement.
+ </para>
+
+ <para>
+ As a result, the modified parsed tree remains usable.
+ </para>
+
+ <para>
+ Examples of non-destructive and useful modifications of the
+ parsed tree are:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ <literal>WHERE</literal>/<literal>ON</literal> clause
+ flattening
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>NOT</literal> elimination
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>LEFT JOIN </literal>elimination, when it can be
+ done based on the constants explicitly specified in the
+ query
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ The rest of modifications are destructive, generally because
+ they are based on actual contents of tables or placeholders.
+ </para>
+
+ <para>
+ Examples of destructive modifications are:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Equality propagation
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Sorting of members of <literal>IN</literal> array for quick
+ evaluation of <literal>IN</literal> expression.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ Destructive modifications are (and should be for all future
+ changes) allocated in a memory root dedicated to execution, are
+ registered in <literal>THD::change_list</literal> and rolled
+ back in the end of each execution. Later the memory root
+ dedicated to execution of a statement will be denoted as the
+ runtime memory root of the statement. Because allocations are
+ done indirectly via <literal>THD::mem_root</literal>,
+ <literal>THD::mem_root</literal> at any given moment of time can
+ point either to the permanent or to the runtime memory root of
+ the statement. Consequently, <literal>THD::mem_root</literal>
+ and <literal>THD::free_list</literal> can be denoted as
+ 'currently active arena' of THD.
+ </para>
+
+ </section>
+
+ <section id="prepared-stored-statement-preparation">
+
+ <title>Preparation of a Prepared Statement</title>
+
+ <para>
+ As mentioned above, <literal>THD</literal> is currently a
+ required argument and the runtime context for every function in
+ the server. Therefore, in order to call the parser and allocate
+ memory in the statement memory root we perform several
+ save-restore steps with <literal>THD::mem_root</literal> and
+ <literal>THD::free_list</literal> (the active arena of
+ <literal>THD</literal>).
+ </para>
+
+ <orderedlist>
+
+ <listitem>
+ <para>
+ In order to parse a statement, we save the currently active
+ arena of <literal>THD</literal> and assign its members from
+ the permanent arena of the statement. This is achieved by
+ calling <literal>THD::set_and_backup_active_arena</literal>.
+ This way <literal>alloc_query</literal> and
+ <literal>yyparse</literal> operate on the permanent arena.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ We don't want the garbage which is created during statement
+ validation to be left in the permanent arena of the
+ statement. For that, after parse but before validation of
+ the statement, we restore the THD arena saved in (1). In
+ other words, we use the arena of <literal>THD</literal> that
+ was active when
+ <literal>Prepared_statement::prepare</literal> was invoked
+ as the runtime arena of the statement when it is validated.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Statement validation is performed in function
+ <literal>check_prepared_statement()</literal>. This function
+ will subsequently call
+ <literal>st_select_lex_unit::prepare()</literal> and
+ <literal>setup_fields()</literal> for the main LEX unit,
+ create <literal>JOINs</literal> for every unit, and call
+ <literal>JOIN::prepare</literal> for every join
+ (<literal>JOINs</literal> in MySQL represents a part of the
+ execution plan). Our prepared statement engine does not save
+ the execution plan in a prepared statement for reuse, and
+ ideally we should not create it at prepare stage. However,
+ currently there is no other way to validate a statement
+ except to call <literal>JOIN::prepare</literal> for all its
+ units.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ During validation we may perform a transformation of the
+ parsed tree. In a clean implementation this would belong to
+ a separate step, but in our case the majority of the server
+ runtime was not refactored to support reexecution of
+ statements, and a permanent transformation of the parsed
+ tree can happen at any moment during validation. Such
+ transformations <emphasis role="bold">absolutely
+ must</emphasis> use the permanent arena of the prepared
+ statement. To make this arena accessible, we save a pointer
+ to it in <literal>thd->stmt_arena</literal> before calling
+ <literal>check_prepared_statement</literal>.
+ </para>
+
+ <para>
+ Later, whenever we need to perform a permanent
+ transformation, we first call
+ <literal>THD::activate_stmt_arena_if_needed</literal> to
+ make the permanent arena active, transform the tree, and
+ restore the runtime arena.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Some parts of the execution do not distinguish between
+ preparation of a prepared statement and its execution and
+ perform destructive optimizations of the parsed tree even
+ during validation. These changes of the parsed tree are
+ recorded in <literal>THD::change_list using</literal> method
+ <literal>THD::register_item_tree_change</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ After the validation is done, we rollback the changes
+ registered in <literal>THD::change_list</literal> and free
+ new items and other memory allocated by destructive
+ transformations.
+ </para>
+ </listitem>
+
+ </orderedlist>
+
+ </section>
+
+ <section id="prepared-stored-statement-execution">
+
+ <title>Execution of a Prepared Statement</title>
+
+ <para>
+ In order to call mysql_execute_command (the function that
+ executes a statement) for a prepared statement and not damage
+ its parse tree, we backup and restore the active Query_arena of
+ THD.
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ We don't want the garbage created during execution to be
+ left in the permanent arena of the statement. To ensure
+ that, every statement is executed in the runtime arena of
+ <literal>THD</literal>. In other words, the arena which was
+ active when <literal>mysql_stmt_execute</literal> was called
+ is used as the runtime arena of the statement during its
+ execution.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Before calling <literal>mysql_stmt_execute</literal>, we
+ <literal>allocate thd->query</literal> with parameter
+ markers ('?') replaced with their values: the new query is
+ allocated in the runtime arena. We'll need this query for
+ general, binary, error and slow logs.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The execution plan created at prepare stage is not saved
+ (see
+ <xref linkend="prepared-stored-statement-preparation"/>),
+ and at execute we simply create a new set of JOINs and then
+ prepare and optimize it. During the first execution of the
+ prepared statement the server may perform non-destructive
+ transformations of statement's parsed tree: normally that
+ would belong to a separate step executed at statement
+ prepare, but once again, this haven't been done in 4.1 or
+ 5.0. Such transformations <emphasis role="bold">absolutely
+ must</emphasis> use the permanent arena of the prepared
+ statement (saved in <literal>thd->stmt_arena</literal>).
+ Whenever we need to perform a permanent transformation, we
+ first call
+ <literal>THD::activate_stmt_arena_if_needed</literal> to
+ make the permanent arena active, transform the tree, and
+ restore the runtime arena. To avoid double transformations
+ in such cases, we track current state of the parsed tree in
+ <literal>Query_arena::state</literal>.
+ </para>
+
+ <para>
+ This state may be one of the following:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ <literal>INITIALIZED</literal> — we're in
+ statement <literal>PREPARE</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>INITIALIZED_FOR_SP</literal> — we're in
+ first execution of a stored procedure statement.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>PREPARED</literal> — we're in first
+ execution of a prepared statement.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>EXECUTED</literal> — we're in a
+ subsequent execution of a prepared statement or a stored
+ procedure statement.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>CONVENTIONAL_EXECUTION</literal> — we're
+ executing a pre-4.1 query.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ One can use helper methods of <literal>Query_arena</literal>
+ to check this state
+ (<literal>is_conventional_execution()</literal>,
+ <literal>is_stmt_prepare()</literal>,
+ <literal>is_stmt_execute()</literal>,
+ <literal>is_stmt_prepare_or_first_sp_execute()</literal>).
+ </para>
+
+ <para>
+ Additionally,
+ <literal>st_select_lex_unit::first_execution</literal>
+ contains a flag for the state of each subquery in a complex
+ statement. A separate variable is needed because not all
+ subqueries may get executed during the first execution of a
+ statement.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Some optimizations damage the parsed tree, e.g. replace
+ leafs and subtrees of items with other items or leave item
+ objects cluttered with runtime data. To allow re-execution
+ of a prepared statement the following mechanisms are
+ currently employed:
+ </para>
+
+ <orderedlist>
+
+ <listitem>
+ <para>
+ A hierarchy of <literal>Item::cleanup()</literal> and
+ <literal>st_select_lex::cleanup()</literal> methods to
+ restore the parsed tree to the condition of
+ right-after-parse. These cleanups are called in
+ <literal>Prepared_statement::cleanup_stmt()</literal>
+ after the statement has been executed.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ In order to roll back destructive transformations of the
+ parsed tree, every replacement of one item with another
+ is registered in <literal>THD::change_list</literal> by
+ using <literal>THD::change_item_tree()</literal>. In the
+ end of execution all such changes are rolled back in
+ reverse order.
+ </para>
+
+ <para>
+ Example:
+ </para>
+
+<programlisting>if (!(fld= new Item_field(from_field)))
+goto error;
+thd->change_item_tree(reference, fld);</programlisting>
+
+ <para>
+ If a transformation is a non-destructive, it should not
+ be registered, but performed only once in the permanent
+ memory root. Additionally, be careful to not supply a
+ pointer to stack as the first argument of
+ <literal>change_item_tree()</literal>; that will lead to
+ stack corruption when a tree is restored.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <literal>AND</literal>/<literal>OR</literal> subtrees of
+ <literal>WHERE</literal> and <literal>ON</literal>
+ clauses are created anew for each execution. It was
+ easier to implement in 4.1, and the approach with change
+ record list used in (b) could not have been used for
+ <literal>AND</literal>/<literal>OR</literal>
+ transformations, because these transformations not only
+ replace one item with another, but also can remove a
+ complete subtree. Leafs of
+ <literal>AND</literal>/<literal>OR</literal> subtrees
+ are not copied by this mechanism because currently they
+ are not damaged by the transformation. For details, see
+ <literal>Item::copy_andor_structure()</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <emphasis role="bold">No</emphasis> other mechanism
+ exists in the server at the moment to allow
+ re-execution. If the code that you're adding transforms
+ the parsed tree, you must use one of the mechanisms
+ described above, or propose and implement a better
+ approach.
+ </para>
+ </listitem>
+
+ </orderedlist>
+ </listitem>
+
+ <listitem>
+ <para>
+ When execution is done, we rollback the damage of the parsed
+ tree.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ </section>
+
+ <section id="prepared-stored-procedure-execution">
+
+ <title>Execution of a Stored Procedure Statement</title>
+
+ <para>
+ Execution of a stored procedure statement is similar to
+ execution of a prepared statement. The few existing exceptions
+ are described below.
+ </para>
+
+ <para>
+ During execution of a stored procedure,
+ <literal>THD::stmt_arena</literal> points to the permanent query
+ arena of the stored procedure. This arena happens to be also the
+ permanent query arena of every instruction of the procedure, as
+ the parser creates all instructions in the same arena. More
+ generally, <literal>THD::stmt_arena</literal> is always set and
+ always points to the permanent arena of a statement. If the
+ statement is a conventional query, then the permanent arena
+ simply points to the runtime arena of the query.
+ </para>
+
+ <para>
+ An own runtime memory root is set up for execution of every
+ stored procedure statement and freed in the end of execution.
+ This is a necessary measure to avoid memory leaks if a stored
+ procedure statement is executed in a loop.
+ </para>
+
+ <para>
+ With regard to the transformations and restoration of the parsed
+ tree, execution of a stored procedure statement follows the path
+ of execution of a prepared statement, with the exception that
+ there is no separate prepare step.
+ <literal>THD::is_first_sp_execute()</literal> is used to
+ determine whether it's the first execution, and in this case
+ non-destructive permanent transformations of the parsed tree are
+ made in the permanent memory root of the statement that is
+ currently being executed.
+ </para>
+
+ <para>
+ During subsequent executions no non-destructive transformations
+ are performed, while all destructive ones are rolled back in the
+ end of execution using the same algorithm as in prepared
+ statements.
+ </para>
+
+ </section>
+
+ </chapter>
+
<chapter id="replication">
<title>Replication</title>
| Thread |
|---|
| • svn commit - mysqldoc@docsrva: r2375 - trunk/internals | mcbrown | 14 Jun |
