List:Falcon Storage Engine« Previous MessageNext Message »
From:Hakan Kuecuekyilmaz Date:November 3 2008 7:58pm
Subject:Weekly Falcon/InnoDB DBT2 Overview 2008-11-03
View as plain text  
Hello benchmark fans,

last week we were busy fixing bugs.

One popular question is: why Falcon outperforms InnoDB on walldorf
in case of a 100 warehouse setup?
Answer: On walldorf I changed the hardware raid controller to
use battery backup, which means that writes end up in the cache
of the raid controller, thus improving IO dramatically. Both,
InnoDB and Falcon gained from that. Falcon showed huge difference
on walldorf with -w100 setup; 1200 --> 3200 NOTPM.

We could fix around 12 bugs:
*  Christopher Powers fixed
  o Bug#36631 Assertion in SerialLogControl::nextRecord
  o Bug#38541 Falcon RecordVersion::thaw assertion ""bytesRestored > 0 || data.record
== NULL"
  o Bug#38567 Falcon crash in Record::getEncodedRecord at Record.cpp:743
  o Bug#38569 Falcon assertion in SRLUpdateIndex::thaw : type == srlUpdateIndex
  o Bug#39694 Crash in StorageTable::setRecord during falcon_chill_thaw test
  o Bug#39695 Crash in SRLUpdateRecords::thaw during falcon_chill_thaw test
  o Bug#40265 Falcon: Concurrent online DROP INDEX of the same key causes MySQL assertion

* Hakan Kuecuekyilmaz fixed
  o Bug#40135 falcon_bug_30124 is too slow, times out in pushbuild

* Kevin Lewis fixed
  o Bug#40123 Falcon crash with falcon_serial_log_buffers=4000

* Olav Sandstaa fixed
  o Bug#38739 Assert in Transaction::initialize during simple SELECT queries
  o Bug#40390 Falcon does not get compiled on Solaris 10 on SPARC

* Sergey Vojtovich fixed
  o Bug#39708 Falcon: non-descriptive error message when creating a big key



News for this week:
- I fixed the PHP problem on lu0009 and the automated PHP/JpGraph
graphing works again.
- nehalem-1 runs ok but until we have a reasonable RAID setup on it
we should not consider the numbers. I just got an approval for a
softraid RAID10 setup with 6 SATA disks.

Change set used:
revno: 2892
revision-id: hky@stripped
parent: hky@stripped
committer: Hakan Kuecuekyilmaz 
branch nick: mysql-6.0-falcon-to-merge
timestamp: Thu 2008-10-30 17:33:46 +0100

caneland.mysql.com[1] numbers can be found here:
https://intranet.mysql.com/~hakank/caneland/weekly_dbt2_overview/weekly_dbt2_overview.php

lu0009.mysql.com[2] numbers can be found here:
https://intranet.mysql.com/~hakank/lu0009/weekly_dbt2_overview/weekly_dbt2_overview.php

nehalem-1.mysql.com[3] numbers can be found here:
https://intranet.mysql.com/~hakank/nehalem-1/weekly_dbt2_overview/weekly_dbt2_overview.php

walldorf@home[4] numbers can be found here:
https://intranet.mysql.com/~hakank/walldorf/weekly_dbt2_overview/weekly_dbt2_overview.php

Brief overview
****************
*** caneland ***
****************
-w10
      -c8      -c16     -c32     -c64     -c99
F/I    47.14%   43.66%   47.04%   46.27%   46.69%

-w20
      -c8      -c16     -c32     -c64     -c128    -c192
F/I    41.98%   38.36%   50.35%   56.09%   60.13%   62.99%

-w100
      -c8      -c16     -c32     -c64     -c128    -c192    -c256    -c499
F/I    55.75%   82.57%   78.04%   67.71%   65.03%   62.93%   66.87%   67.55%

**************
*** lu0009 ***
**************
-w10
      -c8      -c16     -c32     -c64     -c99     
F/I   100.25%   70.95%   47.57%   53.09%   49.57%  

-w20
      -c8      -c16     -c32     -c64     -c128    -c192    
F/I    95.04%   74.39%   56.15%   52.73%   53.08%   55.38%  

-w100
      -c8      -c16     -c32     -c64     -c128    -c192    -c256    -c499    
F/I    79.24%   75.09%   62.18%   59.05%   60.11%   62.89%   64.01%   64.26%  

*****************
*** nehalem-1 ***
*****************
- Please note: following nehalem-1 numbers are from a single SATA disk
setup. We will get a six disk RAID10 softraid setup soon. 
-w10
      -c8      -c16     -c32     -c64     -c99     
F/I    52.63%   55.99%   52.53%   53.46%   54.46%  

-w20
      -c8      -c16     -c32     -c64     -c128    -c192    
F/I    47.59%   51.23%   55.09%   56.79%   56.17%   57.20%  

-w100
      -c8      -c16     -c32     -c64     -c128    -c192    -c256    -c499    
F/I     0.00%    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  

****************
*** walldorf ***
****************
-w10
      -c8      -c16     -c32     -c64     -c99     
F/I    69.22%   68.25%   67.26%   64.31%   63.71%  

-w20
      -c8      -c16     -c32     -c64     -c128    -c192    
F/I    57.63%   57.94%   60.98%   62.40%   62.60%   62.17%  

-w100
      -c8      -c16     -c32     -c64     -c128    -c192    -c256    -c499    
F/I    64.34%   68.40%  111.11%  138.30%  164.44%  166.19%  168.37%  168.03%  

Todo:
  - Remove results from crashed runs in mean value calculation.
  - Add more information from INFORMATION_SCHEMA tables like Falcon's
  system memore usage or summary of active/pending_commit/pending_completion.
  Also talk to Ann to get more insight to Falcon metrics.
  - On walldorf put falcon_serial_log_dir to SATA disk to see
  whether the battery backed up RAID controller is the source of
  huge performance gain we saw lately.

Setup: datadir is on RAID10 and logdir is on separate disk (for InnoDB).

Configuration is:
MYSQLD_CONFIG_BASE="--no-defaults \
  --socket=/tmp/mysql.sock"
MYSQLD_CONFIG_GENERAL="$MYSQLD_CONFIG_BASE \
  --basedir=$FALCON \
  --datadir=$DATA_DIR \
  --tmpdir=/tmp \
  --language=${FALCON}/sql/share/english \
  --key_buffer_size=8M \
  --table_cache=4096 \
  --skip-grant-tables \
  --sort_buffer_size=2M \
  --read_buffer_size=1M \
  --tmp_table_size=67108864 \
  --thread_cache=1024 \
  --query_cache_size=0 \
  --query_cache_type=0 \
  --max_allowed_packet=512M \
  --max_connections=1601 \
  --log-error"
MYSQLD_CONFIG_FALCON="$MYSQLD_CONFIG_GENERAL \
  --skip_innodb \
  --default_storage_engine=Falcon \
  --falcon_io_threads=10 \
  --falcon_record_memory_max=650M \
  --falcon_page_cache_size=1650M \
  --falcon_page_size=8k \
  --falcon_serial_log_dir=$DATA_DIR \
  --falcon_serial_log_buffers=100 \
  --falcon_record_scavenge_threshold=70 \
  --falcon_consistent_read=off"
MYSQLD_CONFIG_INNODB="$MYSQLD_CONFIG_GENERAL \
  --skip_falcon \
  --default_storage_engine=InnoDB \
  --innodb_data_home_dir=$DATA_DIR \
  --innodb_data_file_path=ibdata1:10M:autoextend \
  --innodb_log_group_home_dir=$LOG_DIR \
  --innodb_buffer_pool_size=2300M \
  --innodb_additional_mem_pool_size=20M \
  --innodb_log_file_size=650M \
  --innodb_log_buffer_size=16M \
  --innodb_lock_wait_timeout=50 \
  --innodb_file_io_threads=10 \
  --innodb_flush_log_at_trx_commit=1 \
  --innodb_flush_method=O_DIRECT \
  --innodb_log_files_in_group=2 \
  --innodb_support_xa=0 \
  --innodb_doublewrite=0 \
  --innodb-status-file=0 \
  --skip-innodb-adaptive-hash-index \
  --innodb_max_dirty_pages_pct=70 \
  --innodb_thread_concurrency=0"


[1] caneland.mysql.com SLES10
Quad HexaCore @ 2.6GHz. Intel Xeon with 16MB L2 cache, 32GB RAM, hardware
RAID 10 on 4 x 15k RPM SAS disks.

[2] lu0009.mysql.com SLES10
Quad DualCore @ 3.4Ghz. Intel Xeon (with hyperthreading) with 16MB L2 cache, 8GB RAM,
hardware RAID 10 on 4 x 15k RPM SAS disks. This machine has inofficial Intel CPUs.

[3] nehalem-1.mysql.com Fedora Core 9
Dual DualCore @ 2.4Ghz. Intel Xeon (with SMT) with 16MB L2 cache, 16GB RAM,
Single SATA disk.

[4] walldorf@home RHEL5
Dual DualCore @ 2Ghz. Intel Xeon with 4MB L2 cache, 8GB RAM, hardware
RAID 10 on 4 x 15k RPM SAS disks.


Best regards,

Hakan

-- 
Hakan Küçükyılmaz, Senior Software Engineer DBTG/MySQL +49 160
98953296
Sun Microsystems GmbH     Sonnenallee 1, DE-85551 Kirchheim-Heimstetten
Geschaeftsfuehrer:  Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer
Vorsitz d. Aufs.rat.: Martin Haering   HRB MUC 161028     49.011, 8.376

Thread
Weekly Falcon/InnoDB DBT2 Overview 2008-11-03Hakan Kuecuekyilmaz3 Nov