From: Joey L Date: October 6 2011 11:47am Subject: Re: 4 minute slow on select count(*) from table - myisam type List-Archive: http://lists.mysql.com/mysql/225931 Message-Id: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Just as an fyi - I have other databases and their corresponding apache websites on the same server - performing okay. It seems that apache/mysql server is just having a hard time dealing with the access to those pages that deal with the 9gig table on that particular site. -- Most of the access is done by webcrawlers to the site - so there is a lot of activity occuring on the 9gig tables. thanks mjh On Thu, Oct 6, 2011 at 6:13 AM, Joey L wrote: > guys - i am having such a hard time with this..it is killing me!!! > Sorry - had to vent. > my machine is running an tyan S2912G2NR =A0-- with 2 opterons and 12gig > of memory. I have 2 software raided drives 1gig each. > I run a couple of databases --- my largest table is about 9gig in > size. --it is being accessed a lot. > My my.cnf is as follows: > > # > # The MySQL database server configuration file. > # > # You can copy this to one of: > # - "/etc/mysql/my.cnf" to set global options, > # - "~/.my.cnf" to set user-specific options. > # > # One can use all long options that the program supports. > # Run program with --help to get a list of available options and with > # --print-defaults to see which it would actually understand and use. > # > # For explanations see > # http://dev.mysql.com/doc/mysql/en/server-system-variables.html > > # This will be passed to all mysql clients > # It has been reported that passwords should be enclosed with ticks/quote= s > # escpecially if they contain "#" chars... > # Remember to edit /etc/mysql/debian.cnf when changing the socket locatio= n. > [client] > port =A0 =A0 =A0 =A0 =A0 =A0=3D 3306 > socket =A0 =A0 =A0 =A0 =A0=3D /var/run/mysqld/mysqld.sock > > # Here is entries for some specific programs > # The following values assume you have at least 32M ram > > # This was formally known as [safe_mysqld]. Both versions are currently p= arsed. > [mysqld_safe] > socket =A0 =A0 =A0 =A0 =A0=3D /var/run/mysqld/mysqld.sock > nice =A0 =A0 =A0 =A0 =A0 =A0=3D 0 > > [mysqld] > # > # * Basic Settings > # > user =A0 =A0 =A0 =A0 =A0 =A0=3D mysql > pid-file =A0 =A0 =A0 =A0=3D /var/run/mysqld/mysqld.pid > socket =A0 =A0 =A0 =A0 =A0=3D /var/run/mysqld/mysqld.sock > port =A0 =A0 =A0 =A0 =A0 =A0=3D 3306 > basedir =A0 =A0 =A0 =A0 =3D /usr > datadir =A0 =A0 =A0 =A0 =3D /var/lib/mysql > tmpdir =A0 =A0 =A0 =A0 =A0=3D /tmp > language =A0 =A0 =A0 =A0=3D /usr/share/mysql/english > skip-external-locking > # > # Instead of skip-networking the default is now to listen only on > # localhost which is more compatible and is not less secure. > bind-address =A0 =A0 =A0 =A0 =A0 =A0=3D 127.0.0.1 > # > # * Fine Tuning > # > key_buffer =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D 2G > key_buffer_size =A0 =A0 =A0 =A0 =3D 2G > max_allowed_packet =A0 =A0 =A0=3D 16M > thread_stack =A0 =A0 =A0 =A0 =A0 =A0=3D 192K > thread_cache_size =A0 =A0 =A0 =3D 8 > join_buffer_size =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D 128 > # This replaces the startup script and checks MyISAM tables if needed > # the first time they are touched > myisam-recover =A0 =A0 =A0 =A0 =3D BACKUP > max_connections =A0 =A0 =A0 =A0=3D 100 > table_cache =A0 =A0 =A0 =A0 =A0 =A0=3D 1024 > max_heap_table_size =A0 =A0 =A0 =A0 =A0 =A0 =3D 32M > tmp_table_size =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D 32M > thread_concurrency =A0 =A0 =3D 10 > # > # * Query Cache Configuration > # > query_cache_limit =A0 =A0 =A0 =3D 2M > query_cache_size =A0 =A0 =A0 =A0=3D 16M > # > # * Logging and Replication > # > # Both location gets rotated by the cronjob. > # Be aware that this log type is a performance killer. > # As of 5.1 you can enable the log at runtime! > general_log_file =A0 =A0 =A0 =A0=3D /var/log/mysql/mysql.log > general_log =A0 =A0 =A0 =A0 =A0 =A0 =3D 2 > # > # Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslo= g.cnf. > # > # Here you can see queries with especially long duration > #log_slow_queries =A0 =A0 =A0 =3D /var/log/mysql/mysql-slow.log > #long_query_time =3D 2 > #log-queries-not-using-indexes > # > # The following can be used as easy to replay backup logs or for replicat= ion. > # note: if you are setting up a replication slave, see README.Debian abou= t > # =A0 =A0 =A0 other settings you may need to change. > #server-id =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D 1 > #log_bin =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D /var/log/mysq= l/mysql-bin.log > expire_logs_days =A0 =A0 =A0 =A0=3D 10 > max_binlog_size =A0 =A0 =A0 =A0 =3D 100M > #binlog_do_db =A0 =A0 =A0 =A0 =A0 =3D include_database_name > #binlog_ignore_db =A0 =A0 =A0 =3D include_database_name > # > # * InnoDB > # > # InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. > # Read the manual for more InnoDB related options. There are many! > # > # * Security Features > # > # Read the manual, too, if you want chroot! > # chroot =3D /var/lib/mysql/ > # > # For generating SSL certificates I recommend the OpenSSL GUI "tinyca". > # > # ssl-ca=3D/etc/mysql/cacert.pem > # ssl-cert=3D/etc/mysql/server-cert.pem > # ssl-key=3D/etc/mysql/server-key.pem > > > > [mysqldump] > quick > quote-names > max_allowed_packet =A0 =A0 =A0=3D 64M > > [mysql] > #no-auto-rehash # faster start of mysql but no tab completition > > [isamchk] > key_buffer =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D 16M > > # > # * IMPORTANT: Additional settings that can override those from this file= ! > # =A0 The files must end with '.cnf', otherwise they'll be ignored. > # > !includedir /etc/mysql/conf.d/ > > any thoughts or help would be appricated. > thanks > > > On Mon, Oct 3, 2011 at 1:22 PM, Eric Bergen wrote= : >> Can you run show processlist in another connection while the select >> count(*) query is running and say what the state column is? >> >> On Mon, Oct 3, 2011 at 7:00 AM, Joey L wrote: >>> this is not a real query on the site - it is just a way i am measuring >>> performance on mysql - I do not know if it is such a great way to test. >>> Looking for a better way to get a performance read on my site...do you = have >>> any ?? besides just viewing pages on it. >>> thanks >>> mjh >>> >>> >>> On Mon, Oct 3, 2011 at 9:58 AM, Andr=E9s Tello w= rote: >>> >>>> have you tried >>>> >>>> select count(yourindex) instead of select count(*) ? >>>> >>>> >>>> On Mon, Oct 3, 2011 at 7:53 AM, Joey L wrote: >>>> >>>>> Thanks for the input - >>>>> 1. I will wait 48 hours and see what happens. >>>>> 2. can you tell me what are some performance tests I can do to help m= e >>>>> better tune my server ? >>>>> 3. I am concerned about this table : | w6h8a_sh404sef_urls >>>>> | >>>>> MyISAM | =A0 =A0 =A010 | Dynamic =A0 =A0| 8908402 | =A0 =A0 =A0 =A0 = =A0 =A0174 | =A01551178184 | >>>>> =A0281474976710655 | =A0 2410850304 | =A0 =A0 =A0 =A0 0 | =A0 =A0 =A0= =A08908777 | 2011-09-22 >>>>> 11:16:03 | 2011-10-02 21:17:20 | 2011-10-02 10:12:04 | utf8_general_c= i =A0 | >>>>> =A0NULL | =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | >>>>> what can I do to make it run faster - i did not write the code...but = need >>>>> to >>>>> optimize server to handle this table when it gets larger. =A0It is us= ed for >>>>> url re-writes - so it has a lot of urls. >>>>> thanks >>>>> mjh >>>>> >>>>> On Mon, Oct 3, 2011 at 12:38 AM, Bruce Ferrell >>>> >wrote: >>>>> >>>>> > >>>>> > The meaning is: >>>>> > >>>>> > increase max_connections >>>>> > reduce wait_timeout >>>>> > -- 28800 is wait 8 hours before closing out dead connections >>>>> > same for interactive_timeout >>>>> > >>>>> > >>>>> > increase key_buffer_size (> 7.8G) increase join_buffer_size >>>>> > -- This keeps mysql from having to run to disk constantly for keys >>>>> > -- Key buffer size / total MyISAM indexes: 256.0M/7.8G >>>>> > -- You have a key buffer of 256M and 7.8G of keys >>>>> > >>>>> > join_buffer_size (> 128.0K, or always use indexes with joins) >>>>> > Joins performed without indexes: 23576 of 744k queries. >>>>> > -- You probably want to look at the slow query log. =A0Generalize t= he >>>>> queries >>>>> > and the do an explain on the query. =A0I have seen instances where = a query >>>>> I >>>>> > thought was using an index wasn't and I had to re-write... with hel= p >>>>> from >>>>> > this list :-) =A0Thanks gang! >>>>> > >>>>> > >>>>> > increase tmp_table_size (> 16M) >>>>> > increase max_heap_table_size (> 16M) >>>>> > -- When making adjustments, make tmp_table_size/max_heap_table_size >>>>> equal >>>>> > >>>>> > increase table_cache ( > 1k ) >>>>> > -- Table cache hit rate: 7% (1K open / 14K opened) >>>>> > -- Increase table_cache gradually to avoid file descriptor limits >>>>> > >>>>> > All of the aside, you need to let this run for at least 24 hours. I >>>>> > prefer 48 hours. =A0The first line says mysql has only been running= 9 >>>>> > hours. =A0 You can reset the timeouts interactivly by entering at t= he >>>>> > mysql prompt: >>>>> > >>>>> > set global wait_timeout=3D >>>>> > >>>>> > You can do the same for the interactive_timeout. >>>>> > >>>>> > Setting these values too low will cause long running queries to abo= rt >>>>> > >>>>> > >>>>> > On 10/02/2011 07:02 PM, Joey L wrote: >>>>> > > Variables to adjust: >>>>> > > > =A0 =A0 max_connections (> 100) >>>>> > > > =A0 =A0 wait_timeout (< 28800) >>>>> > > > =A0 =A0 interactive_timeout (< 28800) >>>>> > > > =A0 =A0 key_buffer_size (> 7.8G) >>>>> > > > =A0 =A0 join_buffer_size (> 128.0K, or always use indexes with = joins) >>>>> > > > =A0 =A0 tmp_table_size (> 16M) >>>>> > > > =A0 =A0 max_heap_table_size (> 16M) >>>>> > > > =A0 =A0 table_cache (> 1024) >>>>> > >>>>> > >>>>> > -- >>>>> > MySQL General Mailing List >>>>> > For list archives: http://lists.mysql.com/mysql >>>>> > To unsubscribe: =A0 =A0http://lists.mysql.com/mysql?unsub=3Dmjh2000= @gmail.com >>>>> > >>>>> > >>>>> >>>> >>>> >>> >> >> >> >> -- >> Eric Bergen >> eric.bergen@stripped >> http://www.ebergen.net >> >