List:Internals« Previous MessageNext Message »
From:Sasha Pachev Date:September 3 2001 5:15pm
Subject:Re: Benchmark
View as plain text  
On Monday 03 September 2001 07:06, SDiZ Cheng wrote:
> I have done some benchmark on my machine against mysql-3.23.41 and 
> mysql-4.0 (with laterest changeset from sasha for some test cases) on 
> a dual P-II Xeon 550 with 512M ram on Linux 2.4.4.
> The result is somehow disappointed for me as i can't see much performance 
> improvment in mysql-4.0. And there is some performance degrade in 
> some connect, delete/drop, order by, or key2.
> Is this result expected? or have i done anything wrong in the benchmark?
> 
> --------------
> [client]
> port            = 3306
> socket          = /tmp/mysql.sock
> 
> [mysqld]
> port            = 3306
> socket          = /tmp/mysql.sock
> skip-locking
> set-variable    = key_buffer=16M
> set-variable    = max_allowed_packet=1M
> set-variable    = table_cache=64
> set-variable    = sort_buffer=512K
> set-variable    = net_buffer_length=8K
> set-variable    = myisam_sort_buffer_size=8M
> innodb_data_file_path = ibdata1:400M
> innodb_data_home_dir = /usr/local/mysql4/var/
> innodb_log_group_home_dir = /usr/local/mysql4/var/
> innodb_log_arch_dir = /usr/local/mysql4/var/
> set-variable = innodb_mirrored_log_groups=1
> set-variable = innodb_log_files_in_group=3
> set-variable = innodb_log_file_size=5M
> set-variable = innodb_log_buffer_size=8M
> innodb_flush_log_at_trx_commit=1
> innodb_log_archive=0
> set-variable = innodb_buffer_pool_size=16M
> set-variable = innodb_additional_mem_pool_size=2M
> set-variable = innodb_file_io_threads=4
> set-variable = innodb_lock_wait_timeout=50
> 
> --------------
> This is the result file of the different benchmark tests.
> 
> The number in () after each tests shows how many SQL commands the particular
> test did.  As one test may have many different parameters this gives only
> a rough picture of what was done.  Check the source for more information :)
> 
> Keep in mind that one can't compare benchmarks run with different --cmp
> options. The --cmp options sets the all limits according to the worst
> limit for all server in the benchmark.
> 
> Numbers marked with '+' are estimated according to previous runs because
> the query took longer than a given time-limit to finish. The estimation
> shouldn't be far from the real result thought.
> 
> Numbers marked with '?' contains results that gave wrong result. This can 
only
> be used as an indication of how long it took for the server to produce a 
wrong
> result :)
> 
> Numbers marked with '*' are tests that was run a different number times
> than the test in the first column.  The reason for this is normally that the
> marked test was run with different options that affects the number of tests
> or that the test was from another version of the MySQL benchmarks.
> 
> Hope this will give you some idea how each db is performing at what thing 
....
> Hope you like it .... Luuk & Monty (1997)
> 
> The result logs which where found and the options:
>  1 mysql_fast-Linux_2.4.4_i686             : MySQL 3.23.41 debug --fast 
--socket=/tmp/mysql.sock
>  2 mysql_fast-Linux_2.4.4_mysql4           : MySQL 4.0.0 debug --fast 
--socket=/tmp/mysql.sock

<cut>

You cannot put any value on results of benchmarking a binary compiled with 
debugging even when you want just to compare the results. If one version has 
more DBUG_PRINT statements than the other in some critical section ( which is 
very likely - we never remove DBUG_PRINT() once it is there, and we keep 
adding them as we continue development. When compiled with no debugging, 
DBUG_PRINT() is just a no-op, and does not affect performance at all. 
Additionally, debugging version may have safemalloc and safe-mutex enabled ( 
will be there if you do compile-pentium-debug), which makes things terribly 
slow.

If you want to compare performance, compile both versions without debugging - 
use compile-pentium or compile-pentium-max, not compile-pentium-debug.

-- 
MySQL Development Team
For technical support contracts, visit https://order.mysql.com/
   __  ___     ___ ____  __ 
  /  |/  /_ __/ __/ __ \/ /   Sasha Pachev <sasha@stripped>
 / /|_/ / // /\ \/ /_/ / /__  MySQL AB, http://www.mysql.com/
/_/  /_/\_, /___/\___\_\___/  Provo, Utah, USA
       <___/                  
Thread
BenchmarkSDiZ Cheng3 Sep
  • Re: BenchmarkSasha Pachev3 Sep
  • Re: BenchmarkSDiZ Cheng4 Sep
  • BenchmarkMichael Widenius5 Sep