Author: mcbrown
Date: 2007-02-19 10:40:22 +0100 (Mon, 19 Feb 2007)
New Revision: 4981
Log:
Moving porting chapter from 5.1 manual into the Internals
Added:
trunk/internals/porting.xml
Modified:
trunk/internals/internals.xml
Modified: trunk/internals/internals.xml
===================================================================
--- trunk/internals/internals.xml 2007-02-19 08:23:12 UTC (rev 4980)
+++ trunk/internals/internals.xml 2007-02-19 09:40:22 UTC (rev 4981)
Changed blocks: 1, Lines Added: 2, Lines Deleted: 0; 593 bytes
@@ -95,6 +95,8 @@
<xi:include href="custom-engine.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
+ <xi:include href="porting.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
+
<xi:include href="error-message.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
<xi:include href="mysql-sources.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
Added: trunk/internals/porting.xml
===================================================================
--- trunk/internals/porting.xml (rev 0)
+++ trunk/internals/porting.xml 2007-02-19 09:40:22 UTC (rev 4981)
Changed blocks: 1, Lines Added: 1370, Lines Deleted: 0; 47189 bytes
@@ -0,0 +1,1370 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
+"http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"
+[
+ <!ENTITY % fixedchars.entities SYSTEM "../common/fixedchars.ent">
+ %fixedchars.entities;
+ <!ENTITY % urls.entities SYSTEM "../refman-common/urls.ent">
+ %urls.entities;
+ <!ENTITY % versions.entities SYSTEM "versions.ent">
+ %versions.entities;
+]>
+<appendix id="porting">
+
+ <title>Porting to Other Systems</title>
+
+ <indexterm>
+ <primary>porting</primary>
+ <secondary>to other systems</secondary>
+ </indexterm>
+
+ <para>
+ This appendix helps you port MySQL to other operating systems. Do
+ check the list of currently supported operating systems first. See
+ <xref linkend="which-os"/>. If you have created a new port of MySQL,
+ please let us know so that we can list it here and on our Web site
+ (<ulink url="http://www.mysql.com/"/>), recommending it to other
+ users.
+ </para>
+
+ <para>
+ Note: If you create a new port of MySQL, you are free to copy and
+ distribute it under the GPL license, but it does not make you a
+ copyright holder of MySQL.
+ </para>
+
+ <para>
+ A working POSIX thread library is needed for the server. On Solaris
+ 2.5 we use Sun PThreads (the native thread support in 2.4 and
+ earlier versions is not good enough), on Linux we use LinuxThreads
+ by Xavier Leroy, <email>Xavier.Leroy@stripped</email>.
+ </para>
+
+ <para>
+ The hard part of porting to a new Unix variant without good native
+ thread support is probably to port MIT-pthreads. See
+ <filename>mit-pthreads/README</filename> and Programming POSIX
+ Threads (<ulink url="http://www.humanfactor.com/pthreads/"/>).
+ </para>
+
+ <para>
+ Up to MySQL 4.0.2, the MySQL distribution included a patched version
+ of Chris Provenzano's Pthreads from MIT (see the MIT Pthreads Web
+ page at <ulink url="http://www.mit.edu/afs/sipb/project/pthreads/"/>
+ and a programming introduction at
+ <ulink url="http://www.mit.edu:8001/people/proven/IAP_2000/"/>).
+ These can be used for some operating systems that do not have POSIX
+ threads. See <xref linkend="mit-pthreads"/>.
+ </para>
+
+ <para>
+ It is also possible to use another user level thread package named
+ FSU Pthreads (see
+ <ulink url="http://moss.csc.ncsu.edu/~mueller/pthreads/"/>). This
+ implementation is being used for the SCO port.
+ </para>
+
+ <para>
+ See the <filename>thr_lock.c</filename> and
+ <filename>thr_alarm.c</filename> programs in the
+ <filename>mysys</filename> directory for some tests/examples of
+ these problems.
+ </para>
+
+ <para>
+ Both the server and the client need a working C++ compiler. We use
+ <command>gcc</command> on many platforms. Other compilers that are
+ known to work are SPARCworks, Sun Forte, Irix <command>cc</command>,
+ HP-UX <command>aCC</command>, IBM AIX <command>xlC_r</command>),
+ Intel <command>ecc/icc</command> and Compaq <command>cxx</command>).
+ </para>
+
+ <indexterm>
+ <primary><command>icc</command></primary>
+ <secondary>and MySQL Cluster support></secondary>
+ </indexterm>
+
+ <indexterm>
+ <primary>MySQL Cluster</primary>
+ <secondary>compiling with <command>icc</command></secondary>
+ </indexterm>
+
+ <para>
+ <emphasis role="bold">Important</emphasis>: If you are trying to
+ build MySQL 5.1 with <command>icc</command> on the IA64 platform,
+ and need support for MySQL Cluster, you should first ensure that you
+ are using <command>icc</command> version 9.1.043 or later. (For
+ details, see Bug #21875.)
+ </para>
+
+ <para>
+ To compile only the client use <command>./configure
+ --without-server</command>.
+ </para>
+
+ <para>
+ There is currently no support for only compiling the server, nor is
+ it likely to be added unless someone has a good reason for it.
+ </para>
+
+ <para>
+ If you want/need to change any <filename>Makefile</filename> or the
+ configure script you also need GNU Automake and Autoconf. See
+ <xref linkend="installing-source-tree"/>.
+ </para>
+
+ <para>
+ All steps needed to remake everything from the most basic files.
+ </para>
+
+<programlisting>
+/bin/rm */.deps/*.P
+/bin/rm -f config.cache
+aclocal
+autoheader
+aclocal
+automake
+autoconf
+./configure --with-debug=full --prefix='your installation directory'
+
+# The makefiles generated above need GNU make 3.75 or newer.
+# (called gmake below)
+gmake clean all install init-db
+</programlisting>
+
+ <para>
+ If you run into problems with a new port, you may have to do some
+ debugging of MySQL! See <xref linkend="debugging-server"/>.
+ </para>
+
+ <para>
+ <emphasis role="bold">Note</emphasis>: Before you start debugging
+ <command>mysqld</command>, first get the test programs
+ <literal>mysys/thr_alarm</literal> and
+ <literal>mysys/thr_lock</literal> to work. This ensures that your
+ thread installation has even a remote chance to work!
+ </para>
+
+ <section id="debugging-server">
+
+ <title>Debugging a MySQL Server</title>
+
+ <indexterm>
+ <primary>server</primary>
+ <secondary>debugging</secondary>
+ </indexterm>
+
+ <indexterm>
+ <primary>debugging</primary>
+ <secondary>server</secondary>
+ </indexterm>
+
+ <indexterm>
+ <primary>crash</primary>
+ </indexterm>
+
+ <para>
+ If you are using some functionality that is very new in MySQL, you
+ can try to run <command>mysqld</command> with the
+ <option>--skip-new</option> (which disables all new, potentially
+ unsafe functionality) or with <option>--safe-mode</option> which
+ disables a lot of optimization that may cause problems. See
+ <xref linkend="crashing"/>.
+ </para>
+
+ <para>
+ If <command>mysqld</command> doesn't want to start, you should
+ verify that you don't have any <filename>my.cnf</filename> files
+ that interfere with your setup! You can check your
+ <filename>my.cnf</filename> arguments with <command>mysqld
+ --print-defaults</command> and avoid using them by starting with
+ <command>mysqld --no-defaults ...</command>.
+ </para>
+
+ <para>
+ If <command>mysqld</command> starts to eat up CPU or memory or if
+ it <quote>hangs,</quote> you can use <command>mysqladmin
+ processlist status</command> to find out if someone is executing a
+ query that takes a long time. It may be a good idea to run
+ <command>mysqladmin -i10 processlist status</command> in some
+ window if you are experiencing performance problems or problems
+ when new clients can't connect.
+ </para>
+
+ <para>
+ The command <command>mysqladmin debug</command> dumps some
+ information about locks in use, used memory and query usage to the
+ MySQL log file. This may help solve some problems. This command
+ also provides some useful information even if you haven't compiled
+ MySQL for debugging!
+ </para>
+
+ <para>
+ If the problem is that some tables are getting slower and slower
+ you should try to optimize the table with <literal>OPTIMIZE
+ TABLE</literal> or <command>myisamchk</command>. See
+ <xref linkend="database-administration"/>. You should also check
+ the slow queries with <literal>EXPLAIN</literal>.
+ </para>
+
+ <para>
+ You should also read the OS-specific section in this manual for
+ problems that may be unique to your environment. See
+ <xref linkend="operating-system-specific-notes"/>.
+ </para>
+
+ <section id="compiling-for-debugging">
+
+ <title>Compiling MySQL for Debugging</title>
+
+ <para>
+ If you have some very specific problem, you can always try to
+ debug MySQL. To do this you must configure MySQL with the
+ <option>--with-debug</option> or the
+ <option>--with-debug=full</option> option. You can check whether
+ MySQL was compiled with debugging by doing: <command>mysqld
+ --help</command>. If the <option>--debug</option> flag is listed
+ with the options then you have debugging enabled.
+ <command>mysqladmin ver</command> also lists the
+ <command>mysqld</command> version as <command>mysql ...
+ --debug</command> in this case.
+ </para>
+
+ <para>
+ If you are using <command>gcc</command> or
+ <command>egcs</command>, the recommended
+ <command>configure</command> line is:
+ </para>
+
+<programlisting>
+CC=gcc CFLAGS="-O2" CXX=gcc CXXFLAGS="-O2 -felide-constructors \
+ -fno-exceptions -fno-rtti" ./configure --prefix=/usr/local/mysql \
+ --with-debug --with-extra-charsets=complex
+</programlisting>
+
+ <para>
+ This avoids problems with the <literal>libstdc++</literal>
+ library and with C++ exceptions (many compilers have problems
+ with C++ exceptions in threaded code) and compile a MySQL
+ version with support for all character sets.
+ </para>
+
+ <para>
+ If you suspect a memory overrun error, you can configure MySQL
+ with <option>--with-debug=full</option>, which installs a memory
+ allocation (<literal>SAFEMALLOC</literal>) checker. However,
+ running with <literal>SAFEMALLOC</literal> is quite slow, so if
+ you get performance problems you should start
+ <command>mysqld</command> with the
+ <option>--skip-safemalloc</option> option. This disables the
+ memory overrun checks for each call to
+ <literal>malloc()</literal> and <literal>free()</literal>.
+ </para>
+
+ <para>
+ If <command>mysqld</command> stops crashing when you compile it
+ with <option>--with-debug</option>, you probably have found a
+ compiler bug or a timing bug within MySQL. In this case, you can
+ try to add <option>-g</option> to the <literal>CFLAGS</literal>
+ and <literal>CXXFLAGS</literal> variables above and not use
+ <option>--with-debug</option>. If <command>mysqld</command>
+ dies, you can at least attach to it with <command>gdb</command>
+ or use <command>gdb</command> on the core file to find out what
+ happened.
+ </para>
+
+ <para>
+ When you configure MySQL for debugging you automatically enable
+ a lot of extra safety check functions that monitor the health of
+ <command>mysqld</command>. If they find something
+ <quote>unexpected,</quote> an entry is written to
+ <literal>stderr</literal>, which <command>mysqld_safe</command>
+ directs to the error log! This also means that if you are having
+ some unexpected problems with MySQL and are using a source
+ distribution, the first thing you should do is to configure
+ MySQL for debugging! (The second thing is to send mail to a
+ MySQL mailing list and ask for help. See
+ <xref linkend="mailing-lists"/>. If you believe that you have
+ found a bug, please use the instructions at
+ <xref linkend="bug-reports"/>.
+ </para>
+
+ <para>
+ In the Windows MySQL distribution, <literal>mysqld.exe</literal>
+ is by default compiled with support for trace files.
+ </para>
+
+ </section>
+
+ <section id="making-trace-files">
+
+ <title>Creating Trace Files</title>
+
+ <para>
+ If the <command>mysqld</command> server doesn't start or if you
+ can cause it to crash quickly, you can try to create a trace
+ file to find the problem.
+ </para>
+
+ <para>
+ To do this, you must have a <command>mysqld</command> that has
+ been compiled with debugging support. You can check this by
+ executing <literal>mysqld -V</literal>. If the version number
+ ends with <option>-debug</option>, it's compiled with support
+ for trace files. (On Windows, the debugging server is named
+ <command>mysqld-debug</command> rather than
+ <command>mysqld</command> as of MySQL 4.1.)
+ </para>
+
+ <para>
+ Start the <command>mysqld</command> server with a trace log in
+ <filename>/tmp/mysqld.trace</filename> on Unix or
+ <filename>C:\mysqld.trace</filename> on Windows:
+ </para>
+
+<programlisting>
+shell> <userinput>mysqld --debug</userinput>
+</programlisting>
+
+ <para>
+ On Windows, you should also use the
+ <option>--standalone</option> flag to not start
+ <command>mysqld</command> as a service. In a console window, use
+ this command:
+ </para>
+
+<programlisting>
+C:\> <userinput>mysqld-debug --debug --standalone</userinput>
+</programlisting>
+
+ <para>
+ After this, you can use the <literal>mysql.exe</literal>
+ command-line tool in a second console window to reproduce the
+ problem. You can stop the <command>mysqld</command> server with
+ <command>mysqladmin shutdown</command>.
+ </para>
+
+ <para>
+ Note that the trace file become <emphasis role="bold">very
+ big</emphasis>! If you want to generate a smaller trace file,
+ you can use debugging options something like this:
+ </para>
+
+ <para>
+ <command>mysqld
+ --debug=d,info,error,query,general,where:O,/tmp/mysqld.trace</command>
+ </para>
+
+ <para>
+ This only prints information with the most interesting tags to
+ the trace file.
+ </para>
+
+ <para>
+ If you make a bug report about this, please only send the lines
+ from the trace file to the appropriate mailing list where
+ something seems to go wrong! If you can't locate the wrong
+ place, you can ftp the trace file, together with a full bug
+ report, to <ulink url="&base-url-uploads;"/> so that a MySQL
+ developer can take a look at it.
+ </para>
+
+ <para>
+ The trace file is made with the
+ <emphasis role="bold">DBUG</emphasis> package by Fred Fish. See
+ <xref linkend="the-dbug-package"/>.
+ </para>
+
+ </section>
+
+ <section id="using-gdb-on-mysqld">
+
+ <title>Debugging <command>mysqld</command> under <command>gdb</command></title>
+
+ <indexterm>
+ <primary>gdb</primary>
+ <secondary>using</secondary>
+ </indexterm>
+
+ <para>
+ On most systems you can also start <command>mysqld</command>
+ from <command>gdb</command> to get more information if
+ <command>mysqld</command> crashes.
+ </para>
+
+ <para>
+ With some older <command>gdb</command> versions on Linux you
+ must use <literal>run --one-thread</literal> if you want to be
+ able to debug <command>mysqld</command> threads. In this case,
+ you can only have one thread active at a time. We recommend you
+ to upgrade to gdb 5.1 ASAP as thread debugging works much better
+ with this version!
+ </para>
+
+ <para>
+ NPTL threads (the new thread library on Linux) may cause
+ problems while running <command>mysqld</command> under
+ <command>gdb</command>. Some symptoms are:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ <command>mysqld</command> hangs during startup (before it
+ writes <literal>ready for connections</literal>).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <command>mysqld</command> crashes during a
+ <literal>pthread_mutex_lock()</literal> or
+ <literal>pthread_mutex_unlock()</literal> call.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ In this case, you should set the following environment variable
+ in the shell before starting <command>gdb</command>:
+ </para>
+
+<programlisting>
+LD_ASSUME_KERNEL=2.4.1
+export LD_ASSUME_KERNEL
+</programlisting>
+
+ <para>
+ When running <command>mysqld</command> under
+ <command>gdb</command>, you should disable the stack trace with
+ <option>--skip-stack-trace</option> to be able to catch
+ segfaults within <command>gdb</command>.
+ </para>
+
+ <para>
+ In MySQL 4.0.14 and above you should use the
+ <option>--gdb</option> option to mysqld. This installs an
+ interrupt handler for <literal>SIGINT</literal> (needed to stop
+ <command>mysqld</command> with <literal>^C</literal> to set
+ breakpoints) and disable stack tracing and core file handling.
+ </para>
+
+ <para>
+ It's very hard to debug MySQL under <command>gdb</command> if
+ you do a lot of new connections the whole time as
+ <command>gdb</command> doesn't free the memory for old threads.
+ You can avoid this problem by starting <command>mysqld</command>
+ with <option>--thread_cache_size='max_connections+1'</option>.
+ In most cases just using <option>--thread_cache_size=5'</option>
+ helps a lot!
+ </para>
+
+ <para>
+ If you want to get a core dump on Linux if
+ <command>mysqld</command> dies with a SIGSEGV signal, you can
+ start <command>mysqld</command> with the
+ <option>--core-file</option> option. This core file can be used
+ to make a backtrace that may help you find out why
+ <command>mysqld</command> died:
+ </para>
+
+<programlisting>
+shell> <userinput>gdb mysqld core</userinput>
+gdb> backtrace full
+gdb> quit
+</programlisting>
+
+ <para>
+ See <xref linkend="crashing"/>.
+ </para>
+
+ <para>
+ If you are using <command>gdb</command> 4.17.x or above on
+ Linux, you should install a <filename>.gdb</filename> file, with
+ the following information, in your current directory:
+ </para>
+
+<programlisting>
+set print sevenbit off
+handle SIGUSR1 nostop noprint
+handle SIGUSR2 nostop noprint
+handle SIGWAITING nostop noprint
+handle SIGLWP nostop noprint
+handle SIGPIPE nostop
+handle SIGALRM nostop
+handle SIGHUP nostop
+handle SIGTERM nostop noprint
+</programlisting>
+
+ <para>
+ If you have problems debugging threads with
+ <command>gdb</command>, you should download gdb 5.x and try this
+ instead. The new <command>gdb</command> version has very
+ improved thread handling!
+ </para>
+
+ <para>
+ Here is an example how to debug mysqld:
+ </para>
+
+<programlisting>
+shell> <userinput>gdb /usr/local/libexec/mysqld</userinput>
+gdb> run
+...
+backtrace full # Do this when mysqld crashes
+</programlisting>
+
+ <para>
+ Include the above output in a bug report, which you can file
+ using the instructions in <xref linkend="bug-reports"/>.
+ </para>
+
+ <para>
+ If <command>mysqld</command> hangs you can try to use some
+ system tools like <literal>strace</literal> or
+ <literal>/usr/proc/bin/pstack</literal> to examine where
+ <command>mysqld</command> has hung.
+ </para>
+
+<programlisting>
+strace /tmp/log libexec/mysqld
+</programlisting>
+
+ <indexterm>
+ <primary>DBI->trace</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>trace DBI method</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>DBI_TRACE environment variable</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>Environment variable</primary>
+ <secondary>DBI_TRACE</secondary>
+ </indexterm>
+
+ <para>
+ If you are using the Perl <literal>DBI</literal> interface, you
+ can turn on debugging information by using the
+ <literal>trace</literal> method or by setting the
+ <literal>DBI_TRACE</literal> environment variable.
+ </para>
+
+ </section>
+
+ <section id="using-stack-trace">
+
+ <title>Using a Stack Trace</title>
+
+ <para>
+ On some operating systems, the error log contains a stack trace
+ if <command>mysqld</command> dies unexpectedly. You can use this
+ to find out where (and maybe why) <command>mysqld</command>
+ died. See <xref linkend="error-log"/>. To get a stack trace, you
+ must not compile <command>mysqld</command> with the
+ <option>-fomit-frame-pointer</option> option to gcc. See
+ <xref linkend="compiling-for-debugging"/>.
+ </para>
+
+ <para>
+ If the error file contains something like the following:
+ </para>
+
+<programlisting>
+mysqld got signal 11;
+The manual section 'Debugging a MySQL server' tells you how to
+use a stack trace and/or the core file to produce a readable
+backtrace that may help in finding out why mysqld died
+Attempting backtrace. You can use the following information
+to find out where mysqld died. If you see no messages after
+this, something went terribly wrong...
+stack range sanity check, ok, backtrace follows
+0x40077552
+0x81281a0
+0x8128f47
+0x8127be0
+0x8127995
+0x8104947
+0x80ff28f
+0x810131b
+0x80ee4bc
+0x80c3c91
+0x80c6b43
+0x80c1fd9
+0x80c1686
+</programlisting>
+
+ <para>
+ you can find where <command>mysqld</command> died by doing the
+ following:
+ </para>
+
+ <orderedlist>
+
+ <listitem>
+ <para>
+ Copy the preceding numbers to a file, for example
+ <filename>mysqld.stack</filename>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Make a symbol file for the <command>mysqld</command> server:
+ </para>
+
+<programlisting>
+nm -n libexec/mysqld > /tmp/mysqld.sym
+</programlisting>
+
+ <para>
+ Note that most MySQL binary distributions (except for the
+ "debug" packages, where this information is included inside
+ of the binaries themselves) ship with the above file, named
+ <literal>mysqld.sym.gz</literal>. In this case, you can
+ simply unpack it by doing:
+ </para>
+
+<programlisting>
+gunzip < bin/mysqld.sym.gz > /tmp/mysqld.sym
+</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>
+ Execute <literal>resolve_stack_dump -s /tmp/mysqld.sym -n
+ mysqld.stack</literal>.
+ </para>
+
+ <para>
+ This prints out where <command>mysqld</command> died. If
+ this doesn't help you find out why <command>mysqld</command>
+ died, you should make a bug report and include the output
+ from the above command with the bug report.
+ </para>
+
+ <para>
+ Note however that in most cases it does not help us to just
+ have a stack trace to find the reason for the problem. To be
+ able to locate the bug or provide a workaround, we would in
+ most cases need to know the query that killed
+ <command>mysqld</command> and preferable a test case so that
+ we can repeat the problem! See
+ <xref linkend="bug-reports"/>.
+ </para>
+ </listitem>
+
+ </orderedlist>
+
+ </section>
+
+ <section id="using-log-files">
+
+ <title>Using Server Logs to Find Causes of Errors in <command>mysqld</command></title>
+
+ <para>
+ Note that before starting <command>mysqld</command> with
+ <option>--log</option> you should check all your tables with
+ <command>myisamchk</command>. See
+ <xref linkend="database-administration"/>.
+ </para>
+
+ <para>
+ If <command>mysqld</command> dies or hangs, you should start
+ <command>mysqld</command> with <option>--log</option>. When
+ <command>mysqld</command> dies again, you can examine the end of
+ the log file for the query that killed
+ <command>mysqld</command>.
+ </para>
+
+ <para>
+ If you are using <option>--log</option> without a file name, the
+ log is stored in the database directory as
+ <filename><replaceable>host_name</replaceable>.log</filename> In
+ most cases it is the last query in the log file that killed
+ <command>mysqld</command>, but if possible you should verify
+ this by restarting <command>mysqld</command> and executing the
+ found query from the <command>mysql</command> command-line
+ tools. If this works, you should also test all complicated
+ queries that didn't complete.
+ </para>
+
+ <para>
+ You can also try the command <literal>EXPLAIN</literal> on all
+ <literal>SELECT</literal> statements that takes a long time to
+ ensure that <command>mysqld</command> is using indexes properly.
+ See <xref linkend="explain"/>.
+ </para>
+
+ <para>
+ You can find the queries that take a long time to execute by
+ starting <command>mysqld</command> with
+ <option>--log-slow-queries</option>. See
+ <xref linkend="slow-query-log"/>.
+ </para>
+
+ <para>
+ If you find the text <literal>mysqld restarted</literal> in the
+ error log file (normally named
+ <filename>hostname.err</filename>) you probably have found a
+ query that causes <command>mysqld</command> to fail. If this
+ happens, you should check all your tables with
+ <command>myisamchk</command> (see
+ <xref linkend="database-administration"/>), and test the queries
+ in the MySQL log files to see whether one fails. If you find
+ such a query, try first upgrading to the newest MySQL version.
+ If this doesn't help and you can't find anything in the
+ <literal>mysql</literal> mail archive, you should report the bug
+ to a MySQL mailing list. The mailing lists are described at
+ <ulink url="http://lists.mysql.com/"/>, which also has links to
+ online list archives.
+ </para>
+
+ <para>
+ If you have started <command>mysqld</command> with
+ <literal>myisam-recover</literal>, MySQL automatically checks
+ and tries to repair <literal>MyISAM</literal> tables if they are
+ marked as 'not closed properly' or 'crashed'. If this happens,
+ MySQL writes an entry in the <literal>hostname.err</literal>
+ file <literal>'Warning: Checking table ...'</literal> which is
+ followed by <literal>Warning: Repairing table</literal> if the
+ table needs to be repaired. If you get a lot of these errors,
+ without <command>mysqld</command> having died unexpectedly just
+ before, then something is wrong and needs to be investigated
+ further. See <xref linkend="server-options"/>.
+ </para>
+
+ <para>
+ It is not a good sign if <command>mysqld</command> did die
+ unexpectedly, but in this case, you should not investigate the
+ <literal>Checking table...</literal> messages, but instead try
+ to find out why <command>mysqld</command> died.
+ </para>
+
+ </section>
+
+ <section id="reproducible-test-case">
+
+ <title>Making a Test Case If You Experience Table Corruption</title>
+
+ <para>
+ If you get corrupted tables or if <command>mysqld</command>
+ always fails after some update commands, you can test whether
+ this bug is reproducible by doing the following:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Take down the MySQL daemon (with <command>mysqladmin
+ shutdown</command>).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Make a backup of the tables (to guard against the very
+ unlikely case that the repair does something bad).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Check all tables with <command>myisamchk -s
+ database/*.MYI</command>. Repair any wrong tables with
+ <command>myisamchk -r
+ database/<replaceable>table</replaceable>.MYI</command>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Make a second backup of the tables.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Remove (or move away) any old log files from the MySQL data
+ directory if you need more space.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Start <command>mysqld</command> with
+ <option>--log-bin</option>. See
+ <xref linkend="binary-log"/>. If you want to find a query
+ that crashes <command>mysqld</command>, you should use
+ <option>--log --log-bin</option>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ When you have gotten a crashed table, stop the
+ <literal>mysqld server</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Restore the backup.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Restart the <command>mysqld</command> server
+ <emphasis role="bold">without</emphasis>
+ <option>--log-bin</option>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Re-execute the commands with <command>mysqlbinlog
+ update-log-file | mysql</command>. The update log is saved
+ in the MySQL database directory with the name
+ <literal>hostname-bin.#</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ If the tables are corrupted again or you can get
+ <command>mysqld</command> to die with the above command, you
+ have found reproducible bug that should be easy to fix! FTP
+ the tables and the binary log to
+ <ulink url="&base-url-uploads;"/> and report it in our bugs
+ database using the instructions given in
+ <xref linkend="bug-reports"/>. (Please note that the
+ <filename>/pub/mysql/upload/</filename> FTP directory is not
+ listable, so you'll not see what you've uploaded in your FTP
+ client.) If you are a support customer, you can use the
+ MySQL Customer Support Center
+ <ulink url="https://support.mysql.com/"/> to alert the MySQL
+ team about the problem and have it fixed as soon as
+ possible.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ You can also use the script <literal>mysql_find_rows</literal>
+ to just execute some of the update statements if you want to
+ narrow down the problem.
+ </para>
+
+ </section>
+
+ </section>
+
+ <section id="debugging-client">
+
+ <title>Debugging a MySQL Client</title>
+
+ <indexterm>
+ <primary>debugging</primary>
+ <secondary>client</secondary>
+ </indexterm>
+
+ <indexterm>
+ <primary>clients</primary>
+ <secondary>debugging</secondary>
+ </indexterm>
+
+ <para>
+ To be able to debug a MySQL client with the integrated debug
+ package, you should configure MySQL with
+ <option>--with-debug</option> or
+ <option>--with-debug=full</option>. See
+ <xref linkend="configure-options"/>.
+ </para>
+
+ <indexterm>
+ <primary>MYSQL_DEBUG environment variable</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>Environment variable</primary>
+ <secondary>MYSQL_DEBUG</secondary>
+ </indexterm>
+
+ <para>
+ Before running a client, you should set the
+ <literal>MYSQL_DEBUG</literal> environment variable:
+ </para>
+
+<programlisting>
+shell> <userinput>MYSQL_DEBUG=d:t:O,/tmp/client.trace</userinput>
+shell> <userinput>export MYSQL_DEBUG</userinput>
+</programlisting>
+
+ <para>
+ This causes clients to generate a trace file in
+ <filename>/tmp/client.trace</filename>.
+ </para>
+
+ <para>
+ If you have problems with your own client code, you should attempt
+ to connect to the server and run your query using a client that is
+ known to work. Do this by running <command>mysql</command> in
+ debugging mode (assuming that you have compiled MySQL with
+ debugging on):
+ </para>
+
+<programlisting>
+shell> <userinput>mysql --debug=d:t:O,/tmp/client.trace</userinput>
+</programlisting>
+
+ <para>
+ This provides useful information in case you mail a bug report.
+ See <xref linkend="bug-reports"/>.
+ </para>
+
+ <para>
+ If your client crashes at some 'legal' looking code, you should
+ check that your <filename>mysql.h</filename> include file matches
+ your MySQL library file. A very common mistake is to use an old
+ <filename>mysql.h</filename> file from an old MySQL installation
+ with new MySQL library.
+ </para>
+
+ </section>
+
+ <section id="the-dbug-package">
+
+ <title>The DBUG Package</title>
+
+ <indexterm>
+ <primary>DBUG package</primary>
+ </indexterm>
+
+ <para>
+ The MySQL server and most MySQL clients are compiled with the DBUG
+ package originally created by Fred Fish. When you have configured
+ MySQL for debugging, this package makes it possible to get a trace
+ file of what the program is debugging. See
+ <xref linkend="making-trace-files"/>.
+ </para>
+
+ <para>
+ This section summaries the argument values that you can specify in
+ debug options on the command line for MySQL programs that have
+ been built with debugging support. For more information about
+ programming with the DBUG package, see the DBUG manual in the
+ <filename>dbug</filename> directory of MySQL source distributions.
+ It's best to use a recent distribution for MySQL ¤t-series;
+ to get the most updated DBUG manual.
+ </para>
+
+ <para>
+ You use the debug package by invoking a program with the
+ <option>--debug="..."</option> or the <option>-#...</option>
+ option.
+ </para>
+
+ <para>
+ Most MySQL programs have a default debug string that is used if
+ you don't specify an option to <option>--debug</option>. The
+ default trace file is usually
+ <literal>/tmp/program_name.trace</literal> on Unix and
+ <literal>\program_name.trace</literal> on Windows.
+ </para>
+
+ <para>
+ The debug control string is a sequence of colon-separated fields
+ as follows:
+ </para>
+
+<programlisting>
+<field_1>:<field_2>:...:<field_N>
+</programlisting>
+
+ <para>
+ Each field consists of a mandatory flag character followed by an
+ optional ‘<literal>,</literal>’ and comma-separated
+ list of modifiers:
+ </para>
+
+<programlisting>
+flag[,modifier,modifier,...,modifier]
+</programlisting>
+
+ <para>
+ The currently recognized flag characters are:
+ </para>
+
+ <informaltable>
+ <tgroup cols="2">
+ <colspec colwidth="05*"/>
+ <colspec colwidth="95*"/>
+ <tbody>
+ <row>
+ <entry><emphasis role="bold">Flag</emphasis></entry>
+ <entry><emphasis role="bold">Description</emphasis></entry>
+ </row>
+ <row>
+ <entry><literal>d</literal></entry>
+ <entry>Enable output from DBUG_<N> macros for the current state. May be
+ followed by a list of keywords which selects output only
+ for the DBUG macros with that keyword. An empty list of
+ keywords implies output for all macros.</entry>
+ </row>
+ <row>
+ <entry><literal>D</literal></entry>
+ <entry>Delay after each debugger output line. The argument is the number of
+ tenths of seconds to delay, subject to machine
+ capabilities. For example, <option>-#D,20</option>
+ specifies a delay of two seconds.</entry>
+ </row>
+ <row>
+ <entry><literal>f</literal></entry>
+ <entry>Limit debugging, tracing, and profiling to the list of named functions.
+ Note that a null list disables all functions. The
+ appropriate <literal>d</literal> or <literal>t</literal>
+ flags must still be given; this flag only limits their
+ actions if they are enabled.</entry>
+ </row>
+ <row>
+ <entry><literal>F</literal></entry>
+ <entry>Identify the source file name for each line of debug or trace output.</entry>
+ </row>
+ <row>
+ <entry><literal>i</literal></entry>
+ <entry>Identify the process with the PID or thread ID for each line of debug or
+ trace output.</entry>
+ </row>
+ <row>
+ <entry><literal>g</literal></entry>
+ <entry>Enable profiling. Create a file called <filename>dbugmon.out</filename>
+ containing information that can be used to profile the
+ program. May be followed by a list of keywords that select
+ profiling only for the functions in that list. A null list
+ implies that all functions are considered.</entry>
+ </row>
+ <row>
+ <entry><literal>L</literal></entry>
+ <entry>Identify the source file line number for each line of debug or trace
+ output.</entry>
+ </row>
+ <row>
+ <entry><literal>n</literal></entry>
+ <entry>Print the current function nesting depth for each line of debug or trace
+ output.</entry>
+ </row>
+ <row>
+ <entry><literal>N</literal></entry>
+ <entry>Number each line of debug output.</entry>
+ </row>
+ <row>
+ <entry><literal>o</literal></entry>
+ <entry>Redirect the debugger output stream to the specified file. The default
+ output is <literal>stderr</literal>.</entry>
+ </row>
+ <row>
+ <entry><literal>O</literal></entry>
+ <entry>Like <literal>o</literal>, but the file is really flushed between each
+ write. When needed, the file is closed and reopened
+ between each write.</entry>
+ </row>
+ <row>
+ <entry><literal>p</literal></entry>
+ <entry>Limit debugger actions to specified processes. A process must be
+ identified with the <literal>DBUG_PROCESS</literal> macro
+ and match one in the list for debugger actions to occur.</entry>
+ </row>
+ <row>
+ <entry><literal>P</literal></entry>
+ <entry>Print the current process name for each line of debug or trace output.</entry>
+ </row>
+ <row>
+ <entry><literal>r</literal></entry>
+ <entry>When pushing a new state, do not inherit the previous state's function
+ nesting level. Useful when the output is to start at the
+ left margin.</entry>
+ </row>
+ <row>
+ <entry><literal>S</literal></entry>
+ <entry>Do function <literal>_sanity(_file_,_line_)</literal> at each debugged
+ function until <literal>_sanity()</literal> returns
+ something that differs from 0. (Mostly used with
+ <literal>safemalloc</literal> to find memory leaks)</entry>
+ </row>
+ <row>
+ <entry><literal>t</literal></entry>
+ <entry>Enable function call/exit trace lines. May be followed by a list
+ (containing only one modifier) giving a numeric maximum
+ trace level, beyond which no output occurs for either
+ debugging or tracing macros. The default is a compile time
+ option.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+
+ <para>
+ Some examples of debug control strings that might appear on a
+ shell command line (the <option>-#</option> is typically used to
+ introduce a control string to an application program) are:
+ </para>
+
+<programlisting>
+-#d:t
+-#d:f,main,subr1:F:L:t,20
+-#d,input,output,files:n
+-#d:t:i:O,\\mysqld.trace
+</programlisting>
+
+ <para>
+ In MySQL, common tags to print (with the <literal>d</literal>
+ option) are <literal>enter</literal>, <literal>exit</literal>,
+ <literal>error</literal>, <literal>warning</literal>,
+ <literal>info</literal>, and <literal>loop</literal>.
+ </para>
+
+ </section>
+
+ <section id="rts-threads">
+
+ <title>Comments about RTS Threads</title>
+
+ <indexterm>
+ <primary>RTS-threads</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>threads</primary>
+ <secondary>RTS</secondary>
+ </indexterm>
+
+ <para>
+ I have tried to use the RTS thread packages with MySQL but
+ stumbled on the following problems:
+ </para>
+
+ <para>
+ They use old versions of many POSIX calls and it is very tedious
+ to make wrappers for all functions. I am inclined to think that it
+ would be easier to change the thread libraries to the newest POSIX
+ specification.
+ </para>
+
+ <para>
+ Some wrappers are currently written. See
+ <filename>mysys/my_pthread.c</filename> for more info.
+ </para>
+
+ <para>
+ At least the following should be changed:
+ </para>
+
+ <para>
+ <literal>pthread_get_specific</literal> should use one argument.
+ <literal>sigwait</literal> should take two arguments. A lot of
+ functions (at least <literal>pthread_cond_wait</literal>,
+ <literal>pthread_cond_timedwait()</literal>) should return the
+ error code on error. Now they return -1 and set
+ <literal>errno</literal>.
+ </para>
+
+ <para>
+ Another problem is that user-level threads use the
+ <literal>ALRM</literal> signal and this aborts a lot of functions
+ (<literal>read</literal>, <literal>write</literal>,
+ <literal>open</literal>...). MySQL should do a retry on interrupt
+ on all of these but it is not that easy to verify it.
+ </para>
+
+ <para>
+ The biggest unsolved problem is the following:
+ </para>
+
+ <para>
+ To get thread-level alarms I changed
+ <filename>mysys/thr_alarm.c</filename> to wait between alarms with
+ <literal>pthread_cond_timedwait()</literal>, but this aborts with
+ error <literal>EINTR</literal>. I tried to debug the thread
+ library as to why this happens, but couldn't find any easy
+ solution.
+ </para>
+
+ <para>
+ If someone wants to try MySQL with RTS threads I suggest the
+ following:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ Change functions MySQL uses from the thread library to POSIX.
+ This shouldn't take that long.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Compile all libraries with the
+ <option>-DHAVE_rts_threads</option>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Compile <literal>thr_alarm</literal>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ If there are some small differences in the implementation,
+ they may be fixed by changing
+ <filename>my_pthread.h</filename> and
+ <filename>my_pthread.c</filename>.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Run <literal>thr_alarm</literal>. If it runs without any
+ <quote>warning,</quote> <quote>error,</quote> or aborted
+ messages, you are on the right track. Here is a successful run
+ on Solaris:
+ </para>
+
+<programlisting>
+Main thread: 1
+Thread 0 (5) started
+Thread: 5 Waiting
+process_alarm
+Thread 1 (6) started
+Thread: 6 Waiting
+process_alarm
+process_alarm
+thread_alarm
+Thread: 6 Slept for 1 (1) sec
+Thread: 6 Waiting
+process_alarm
+process_alarm
+thread_alarm
+Thread: 6 Slept for 2 (2) sec
+Thread: 6 Simulation of no alarm needed
+Thread: 6 Slept for 0 (3) sec
+Thread: 6 Waiting
+process_alarm
+process_alarm
+thread_alarm
+Thread: 6 Slept for 4 (4) sec
+Thread: 6 Waiting
+process_alarm
+thread_alarm
+Thread: 5 Slept for 10 (10) sec
+Thread: 5 Waiting
+process_alarm
+process_alarm
+thread_alarm
+Thread: 6 Slept for 5 (5) sec
+Thread: 6 Waiting
+process_alarm
+process_alarm
+
+...
+thread_alarm
+Thread: 5 Slept for 0 (1) sec
+end
+</programlisting>
+ </listitem>
+
+ </itemizedlist>
+
+ </section>
+
+ <section id="thread-packages">
+
+ <title>Differences Between Thread Packages</title>
+
+ <indexterm>
+ <primary>thread packages</primary>
+ <secondary>differences between</secondary>
+ </indexterm>
+
+ <para>
+ MySQL is very dependent on the thread package used. So when
+ choosing a good platform for MySQL, the thread package is very
+ important.
+ </para>
+
+ <para>
+ There are at least three types of thread packages:
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ User threads in a single process. Thread switching is managed
+ with alarms and the threads library manages all
+ non-thread-safe functions with locks. Read, write and select
+ operations are usually managed with a thread-specific select
+ that switches to another thread if the running threads have to
+ wait for data. If the user thread packages are integrated in
+ the standard libs (FreeBSD and BSDI threads) the thread
+ package requires less overhead than thread packages that have
+ to map all unsafe calls (MIT-pthreads, FSU Pthreads and RTS
+ threads). In some environments (for example, SCO), all system
+ calls are thread-safe so the mapping can be done very easily
+ (FSU Pthreads on SCO). Downside: All mapped calls take a
+ little time and it's quite tricky to be able to handle all
+ situations. There are usually also some system calls that are
+ not handled by the thread package (like MIT-pthreads and
+ sockets). Thread scheduling isn't always optimal.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ User threads in separate processes. Thread switching is done
+ by the kernel and all data are shared between threads. The
+ thread package manages the standard thread calls to allow
+ sharing data between threads. LinuxThreads is using this
+ method. Downside: Lots of processes. Thread creating is slow.
+ If one thread dies the rest are usually left hanging and you
+ must kill them all before restarting. Thread switching is
+ somewhat expensive.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Kernel threads. Thread switching is handled by the thread
+ library or the kernel and is very fast. Everything is done in
+ one process, but on some systems, <command>ps</command> may
+ show the different threads. If one thread aborts, the whole
+ process aborts. Most system calls are thread-safe and should
+ require very little overhead. Solaris, HP-UX, AIX and OSF/1
+ have kernel threads.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ In some systems kernel threads are managed by integrating user
+ level threads in the system libraries. In such cases, the thread
+ switching can only be done by the thread library and the kernel
+ isn't really <quote>thread aware.</quote>
+ </para>
+
+ </section>
+
+</appendix>
| Thread |
|---|
| • svn commit - mysqldoc@docsrva: r4981 - trunk/internals | mcbrown | 19 Feb |
