List:General Discussion« Previous MessageNext Message »
From:Joey L Date:October 6 2011 10:13am
Subject:Re: 4 minute slow on select count(*) from table - myisam type
View as plain text  
guys - i am having such a hard time with this..it is killing me!!!
Sorry - had to vent.
my machine is running an tyan S2912G2NR  -- with 2 opterons and 12gig
of memory. I have 2 software raided drives 1gig each.
I run a couple of databases --- my largest table is about 9gig in
size. --it is being accessed a lot.
My my.cnf is as follows:

#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port		= 3306
socket		= /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket		= /var/run/mysqld/mysqld.sock
nice		= 0

[mysqld]
#
# * Basic Settings
#
user		= mysql
pid-file	= /var/run/mysqld/mysqld.pid
socket		= /var/run/mysqld/mysqld.sock
port		= 3306
basedir		= /usr
datadir		= /var/lib/mysql
tmpdir		= /tmp
language	= /usr/share/mysql/english
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address		= 127.0.0.1
#
# * Fine Tuning
#
key_buffer		= 2G
key_buffer_size		= 2G
max_allowed_packet	= 16M
thread_stack		= 192K
thread_cache_size       = 8
join_buffer_size		= 128
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover         = BACKUP
max_connections        = 100
table_cache            = 1024
max_heap_table_size		= 32M
tmp_table_size			= 32M
thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit	= 2M
query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
general_log_file        = /var/log/mysql/mysql.log
general_log             = 2
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# Here you can see queries with especially long duration
#log_slow_queries	= /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id		= 1
#log_bin			= /var/log/mysql/mysql-bin.log
expire_logs_days	= 10
max_binlog_size         = 100M
#binlog_do_db		= include_database_name
#binlog_ignore_db	= include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem



[mysqldump]
quick
quote-names
max_allowed_packet	= 64M

[mysql]
#no-auto-rehash	# faster start of mysql but no tab completition

[isamchk]
key_buffer		= 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

any thoughts or help would be appricated.
thanks


On Mon, Oct 3, 2011 at 1:22 PM, Eric Bergen <eric.bergen@stripped> wrote:
> Can you run show processlist in another connection while the select
> count(*) query is running and say what the state column is?
>
> On Mon, Oct 3, 2011 at 7:00 AM, Joey L <mjh2000@stripped> wrote:
>> this is not a real query on the site - it is just a way i am measuring
>> performance on mysql - I do not know if it is such a great way to test.
>> Looking for a better way to get a performance read on my site...do you have
>> any ?? besides just viewing pages on it.
>> thanks
>> mjh
>>
>>
>> On Mon, Oct 3, 2011 at 9:58 AM, Andrés Tello <mr.criptos@stripped>
> wrote:
>>
>>> have you tried
>>>
>>> select count(yourindex) instead of select count(*) ?
>>>
>>>
>>> On Mon, Oct 3, 2011 at 7:53 AM, Joey L <mjh2000@stripped> wrote:
>>>
>>>> Thanks for the input -
>>>> 1. I will wait 48 hours and see what happens.
>>>> 2. can you tell me what are some performance tests I can do to help me
>>>> better tune my server ?
>>>> 3. I am concerned about this table : | w6h8a_sh404sef_urls
>>>> |
>>>> MyISAM |      10 | Dynamic    | 8908402 |  
>          174 |  1551178184 |
>>>>  281474976710655 |   2410850304 |         0
> |        8908777 | 2011-09-22
>>>> 11:16:03 | 2011-10-02 21:17:20 | 2011-10-02 10:12:04 | utf8_general_ci
>   |
>>>>  NULL |                |
>                          
>         |
>>>> what can I do to make it run faster - i did not write the code...but
> need
>>>> to
>>>> optimize server to handle this table when it gets larger.  It is
> used for
>>>> url re-writes - so it has a lot of urls.
>>>> thanks
>>>> mjh
>>>>
>>>> On Mon, Oct 3, 2011 at 12:38 AM, Bruce Ferrell <bferrell@stripped
>>>> >wrote:
>>>>
>>>> >
>>>> > The meaning is:
>>>> >
>>>> > increase max_connections
>>>> > reduce wait_timeout
>>>> > -- 28800 is wait 8 hours before closing out dead connections
>>>> > same for interactive_timeout
>>>> >
>>>> >
>>>> > increase key_buffer_size (> 7.8G) increase join_buffer_size
>>>> > -- This keeps mysql from having to run to disk constantly for keys
>>>> > -- Key buffer size / total MyISAM indexes: 256.0M/7.8G
>>>> > -- You have a key buffer of 256M and 7.8G of keys
>>>> >
>>>> > join_buffer_size (> 128.0K, or always use indexes with joins)
>>>> > Joins performed without indexes: 23576 of 744k queries.
>>>> > -- You probably want to look at the slow query log.  Generalize
> the
>>>> queries
>>>> > and the do an explain on the query.  I have seen instances
> where a query
>>>> I
>>>> > thought was using an index wasn't and I had to re-write... with
> help
>>>> from
>>>> > this list :-)  Thanks gang!
>>>> >
>>>> >
>>>> > increase tmp_table_size (> 16M)
>>>> > increase max_heap_table_size (> 16M)
>>>> > -- When making adjustments, make tmp_table_size/max_heap_table_size
>>>> equal
>>>> >
>>>> > increase table_cache ( > 1k )
>>>> > -- Table cache hit rate: 7% (1K open / 14K opened)
>>>> > -- Increase table_cache gradually to avoid file descriptor limits
>>>> >
>>>> > All of the aside, you need to let this run for at least 24 hours. I
>>>> > prefer 48 hours.  The first line says mysql has only been
> running 9
>>>> > hours.   You can reset the timeouts interactivly by entering at
> the
>>>> > mysql prompt:
>>>> >
>>>> > set global wait_timeout=<some value>
>>>> >
>>>> > You can do the same for the interactive_timeout.
>>>> >
>>>> > Setting these values too low will cause long running queries to
> abort
>>>> >
>>>> >
>>>> > On 10/02/2011 07:02 PM, Joey L wrote:
>>>> > > Variables to adjust:
>>>> > > >     max_connections (> 100)
>>>> > > >     wait_timeout (< 28800)
>>>> > > >     interactive_timeout (< 28800)
>>>> > > >     key_buffer_size (> 7.8G)
>>>> > > >     join_buffer_size (> 128.0K, or always use
> indexes with joins)
>>>> > > >     tmp_table_size (> 16M)
>>>> > > >     max_heap_table_size (> 16M)
>>>> > > >     table_cache (> 1024)
>>>> >
>>>> >
>>>> > --
>>>> > MySQL General Mailing List
>>>> > For list archives: http://lists.mysql.com/mysql
>>>> > To unsubscribe:    http://lists.mysql.com/mysql?unsub=1
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>
>
>
> --
> Eric Bergen
> eric.bergen@stripped
> http://www.ebergen.net
>
Thread
4 minute slow on select count(*) from table - myisam typeJoey L2 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore2 Oct
    • Re: 4 minute slow on select count(*) from table - myisam typeJoey L2 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeBruce Ferrell2 Oct
Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore2 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeJoey L3 Oct
    • Re: 4 minute slow on select count(*) from table - myisam typeJoey L3 Oct
      • Re: 4 minute slow on select count(*) from table - myisam typeBruce Ferrell3 Oct
        • Re: 4 minute slow on select count(*) from table - myisam typeJoey L3 Oct
          • Re: 4 minute slow on select count(*) from table - myisam typeAndrĂ©s Tello3 Oct
            • Re: 4 minute slow on select count(*) from table - myisam typeJoey L3 Oct
              • Re: 4 minute slow on select count(*) from table - myisam typeEric Bergen3 Oct
                • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
                  • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
                    • Re: 4 minute slow on select count(*) from table - myisam typeJohnny Withers6 Oct
                      • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
                        • Re: 4 minute slow on select count(*) from table - myisam typeJohnny Withers6 Oct
                        • Re: 4 minute slow on select count(*) from table - myisam typeJohan De Meersman6 Oct
                        • Re: 4 minute slow on select count(*) from table - myisam typeRik Wasmus6 Oct
                          • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
                            • Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore6 Oct
                              • Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore6 Oct
                                • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
                                  • Re: 4 minute slow on select count(*) from table - myisam typeMichael Dykman6 Oct
Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore6 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeJoey L6 Oct
Re: 4 minute slow on select count(*) from table - myisam typeJohnny Withers6 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeJohan De Meersman7 Oct
Re: 4 minute slow on select count(*) from table - myisam typeJan Steinman7 Oct
  • Re: 4 minute slow on select count(*) from table - myisam typeJoey L7 Oct
    • Re: 4 minute slow on select count(*) from table - myisam typeAndrew Moore7 Oct