> <cut>
>
> You cannot put any value on results of benchmarking a binary compiled with
> debugging even when you want just to compare the results. If one version has
> more DBUG_PRINT statements than the other in some critical section ( which is
> very likely - we never remove DBUG_PRINT() once it is there, and we keep
> adding them as we continue development. When compiled with no debugging,
> DBUG_PRINT() is just a no-op, and does not affect performance at all.
> Additionally, debugging version may have safemalloc and safe-mutex enabled (
> will be there if you do compile-pentium-debug), which makes things terribly
> slow.
>
> If you want to compare performance, compile both versions without debugging -
> use compile-pentium or compile-pentium-max, not compile-pentium-debug.
Hi,
I have try another benchmark against MySQL 3.23.41 and MySQL 4.0 on the
same machine, with another set of configure option. Looks more reasonable now.
Let me post it here for your reference:
-=-=-=-=-=-=-=-=
This is the result file of the different benchmark tests.
The number in () after each tests shows how many SQL commands the particular
test did. As one test may have many different parameters this gives only
a rough picture of what was done. Check the source for more information :)
Keep in mind that one can't compare benchmarks run with different --cmp
options. The --cmp options sets the all limits according to the worst
limit for all server in the benchmark.
Numbers marked with '+' are estimated according to previous runs because
the query took longer than a given time-limit to finish. The estimation
shouldn't be far from the real result thought.
Numbers marked with '?' contains results that gave wrong result. This can only
be used as an indication of how long it took for the server to produce a wrong
result :)
Numbers marked with '*' are tests that was run a different number times
than the test in the first column. The reason for this is normally that the
marked test was run with different options that affects the number of tests
or that the test was from another version of the MySQL benchmarks.
Hope this will give you some idea how each db is performing at what thing ....
Hope you like it .... Luuk & Monty (1997)
The result logs which where found and the options:
1 mysql-MySQL3.23 : MySQL 3.23.41
2 mysql-MySQL4 : MySQL 4.0.0
=====================================================
Operation | 1| 2|
|mysql-M|mysql-M|
-----------------------------------------------------
Results per test in seconds: |
-----------------------------------------------------
ATIS | 57.00| 59.00|
alter-table | 535.00| 485.00|
big-tables | 45.00| 45.00|
connect | 96.00| 95.00|
create | 395.00| 390.00|
insert |3095.00|3005.00|
select |2259.00|+2092.00|
wisconsin | 19.00| 19.00|
-----------------------------------------------------
The results per operation: |
-----------------------------------------------------
alter_table_add (992) | 295.00| 268.00|
alter_table_drop (496) | 229.00| 206.00|
connect (10000) | 14.00| 15.00|
connect+select_1_row (10000) | 17.00| 17.00|
connect+select_simple (10000) | 16.00| 16.00|
count (100) | 72.00| 59.00|
count_distinct (1000) | 80.00| 57.00|
count_distinct_2 (1000) | 84.00| 70.00|
count_distinct_big (120) | 147.00| 92.00|
count_distinct_group (1000) | 117.00| 90.00|
count_distinct_group_on_key (1000) | 78.00| 79.00|
count_distinct_group_on_key_parts (1| 116.00| 89.00|
count_distinct_key_prefix (1000) | 68.00| 44.00|
count_group_on_key_parts (1000) | 74.00| 81.00|
count_on_key (50100) | 752.00|+761.00|
create+drop (10000) | 15.00| 15.00|
create_MANY_tables (10000) | 344.00| 338.00|
create_index (8) | 6.00| 5.00|
create_key+drop (10000) | 20.00| 20.00|
create_table (31) | 0.00| 0.00|
delete_all (12) | 17.00| 16.00|
delete_all_many_keys (1) | 84.00| 76.00|
delete_big (1) | 0.00| 0.00|
delete_big_many_keys (128) | 84.00| 76.00|
delete_key (10000) | 5.00| 4.00|
drop_index (8) | 5.00| 6.00|
drop_table (28) | 0.00| 0.00|
drop_table_when_MANY_tables (10000) | 10.00| 10.00|
insert (350768) | 147.00| 146.00|
insert_duplicates (100000) | 35.00| 34.00|
insert_key (100000) | 143.00| 134.00|
insert_many_fields (2000) | 13.00| 13.00|
insert_select_1_key (1) | 9.00| 9.00|
insert_select_2_keys (1) | 11.00| 12.00|
min_max (60) | 38.00| 40.00|
min_max_on_key (85000) | 284.00| 298.00|
multiple_value_insert (100000) | 9.00| 9.00|
order_by_big (10) | 61.00| 63.00|
order_by_big_key (10) | 45.00| 46.00|
order_by_big_key2 (10) | 44.00| 44.00|
order_by_big_key_desc (10) | 46.00| 48.00|
order_by_big_key_diff (10) | 57.00| 59.00|
order_by_big_key_prefix (10) | 43.00| 44.00|
order_by_key2_diff (500) | 7.00| 7.00|
order_by_key_prefix (500) | 4.00| 4.00|
order_by_range (500) | 6.00| 7.00|
outer_join (10) | 96.00| 98.00|
outer_join_found (10) | 92.00| 93.00|
outer_join_not_found (500) | 64.00| 63.00|
outer_join_on_key (10) | 73.00| 74.00|
select_1_row (10000) | 3.00| 3.00|
select_2_rows (10000) | 5.00| 4.00|
select_big (80) | 44.00| 45.00|
select_big_str (10000) | 35.00| 34.00|
select_column+column (10000) | 4.00| 4.00|
select_diff_key (500) | 255.00| 260.00|
select_distinct (800) | 15.00| 15.00|
select_group (2911) | 92.00| 93.00|
select_group_when_MANY_tables (10000| 6.00| 7.00|
select_join (100) | 3.00| 3.00|
select_key (200000) | 167.00| 157.00|
select_key2 (200000) | 174.00| 165.00|
select_key2_return_key (200000) | 166.00| 152.00|
select_key2_return_prim (200000) | 168.00| 158.00|
select_key_prefix (200000) | 172.00| 165.00|
select_key_prefix_join (100) | 16.00| 17.00|
select_key_return_key (200000) | 155.00| 153.00|
select_many_fields (2000) | 31.00| 32.00|
select_query_cache (10000) | 177.00| 161.00|
select_query_cache2 (10000) | 177.00| 161.00|
select_range (410) | 265.00| 257.00|
select_range_key2 (25010) | 25.00| 24.00|
select_range_prefix (25010) | 26.00| 25.00|
select_simple (10000) | 2.00| 2.00|
select_simple_join (500) | 2.00| 2.00|
update_big (10) | 38.00| 29.00|
update_of_key (50000) | 28.00| 26.00|
update_of_key_big (501) | 33.00| 33.00|
update_of_primary_key_many_keys (256| 48.00| 48.00|
update_with_key (300000) | 126.00| 122.00|
update_with_key_prefix (100000) | 41.00| 41.00|
wisc_benchmark (114) | 6.00| 6.00|
-----------------------------------------------------
TOTALS |6581.00|+6259.00|
=====================================================
