From: Hakan Kuecuekyilmaz Date: May 4 2009 6:51am Subject: Weekly Falcon/InnoDB DBT2 Overview 2009-05-04 List-Archive: http://lists.mysql.com/falcon/710 Message-Id: <1241419872.6233.530.camel@lu0011> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Hello benchmark fans, 100 warehouse runs did not run properly on caneland, lu0009, and nehalem-1 News for this week: * I tested a patch from Kevin on caneland, which gave us 40 - 60% performance gain for Falcon. * caneland did not run 100 warehouses properly due to OS issues. caneland.mysql.com[1] numbers can be found here: https://intranet.mysql.com/~hakank/caneland/weekly_dbt2_overview/weekly_dbt2_overview.php lu0009.mysql.com[2] numbers can be found here: https://intranet.mysql.com/~hakank/lu0009/weekly_dbt2_overview/weekly_dbt2_overview.php nehalem-1.mysql.com[3] numbers can be found here: https://intranet.mysql.com/~hakank/nehalem-1/weekly_dbt2_overview/weekly_dbt2_overview.php walldorf@home[4] numbers can be found here: https://intranet.mysql.com/~hakank/walldorf/weekly_dbt2_overview/weekly_dbt2_overview.php Brief overview **************** *** caneland *** **************** -w10 -c8 -c16 -c32 -c64 -c99 F/I 37.36% 29.62% 31.60% 30.85% 29.62% -w20 -c8 -c16 -c32 -c64 -c128 -c192 F/I 37.35% 32.11% 35.53% 34.05% 30.23% 32.35% -w100 -c8 -c16 -c32 -c64 -c128 -c192 -c256 -c499 F/I 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% ************** *** lu0009 *** ************** -w10 -c8 -c16 -c32 -c64 -c99 F/I 84.87% 53.57% 38.71% 37.00% 35.49% -w20 -c8 -c16 -c32 -c64 -c128 -c192 F/I 65.80% 59.37% 53.12% 50.59% 52.55% 62.81% -w100 -c8 -c16 -c32 -c64 -c128 -c192 -c256 -c499 F/I 76.07% 74.53% 0.00% 0.00% 54.93% 0.00% 53.29% 0.00% ***************** *** nehalem-1 *** ***************** -w10 -c8 -c16 -c32 -c64 -c99 F/I 63.00% 66.20% 60.52% 60.59% 67.04% -w20 -c8 -c16 -c32 -c64 -c128 -c192 F/I 35.00% 46.67% 45.73% 46.03% 55.91% 66.38% -w100 -c8 -c16 -c32 -c64 -c128 -c192 -c256 -c499 F/I 0.00% 0.00% 0.00% 0.00% 95.98% 0.00% 0.00% 99.92% **************** *** walldorf *** **************** -w10 -c8 -c16 -c32 -c64 -c99 F/I 61.35% 57.90% 55.94% 54.68% 53.46% -w20 -c8 -c16 -c32 -c64 -c128 -c192 F/I 54.27% 50.11% 53.65% 51.93% 52.34% 53.15% -w100 -c8 -c16 -c32 -c64 -c128 -c192 -c256 -c499 F/I 93.83% 113.25% 131.61% 133.05% 137.52% 143.66% 151.36% 144.15% Todo: - Put temporary results directory of DBT2 to local disk instead of storing the data NFS.  - Remove results from crashed runs in mean value calculation. - Add more information from INFORMATION_SCHEMA tables like Falcon's system memory usage or summary of active/pending_commit/pending_completion. Also talk to Ann to get more insight to Falcon metrics. - On walldorf put falcon_serial_log_dir to SATA disk to see whether the battery backed up RAID controller is the source of huge performance gain we saw lately. - Get a Solaris SPARC and Intel/AMD machine for DBT2. Change set used: revno: 2700 revision-id: hky@stripped parent: hky@stripped committer: Hakan Kuecuekyilmaz branch nick: mysql-6.0-falcon-to-merge timestamp: Thu 2009-04-30 12:33:34 +0200 Setup: datadir is on RAID10 and logdir is on separate disk (for InnoDB). Configuration is: MYSQLD_CONFIG_BASE="--no-defaults \ --socket=/tmp/mysql.sock" MYSQLD_CONFIG_GENERAL="$MYSQLD_CONFIG_BASE \ --basedir=$FALCON \ --datadir=$DATA_DIR \ --tmpdir=/tmp \ --language=${FALCON}/sql/share/english \ --key_buffer_size=8M \ --skip-grant-tables \ --sort_buffer_size=2M \ --read_buffer_size=1M \ --table_open_cache=5120 \ --tmp_table_size=67108864 \ --thread_cache=1024 \ --query_cache_size=0 \ --query_cache_type=0 \ --max_allowed_packet=512M \ --max_connections=707 \ --log-error" MYSQLD_CONFIG_FALCON="$MYSQLD_CONFIG_GENERAL \ --skip_innodb \ --default_storage_engine=Falcon \ --falcon_io_threads=10 \ --falcon_record_memory_max=650M \ --falcon_page_cache_size=1650M \ --falcon_page_size=8k \ --falcon_serial_log_dir=$DATA_DIR \ --falcon_serial_log_buffers=100 \ --falcon_consistent_read=off" MYSQLD_CONFIG_INNODB="$MYSQLD_CONFIG_GENERAL \ --skip_falcon \ --default_storage_engine=InnoDB \ --innodb_data_home_dir=$DATA_DIR \ --innodb_data_file_path=ibdata1:10M:autoextend \ --innodb_log_group_home_dir=$LOG_DIR \ --innodb_buffer_pool_size=2300M \ --innodb_additional_mem_pool_size=20M \ --innodb_log_file_size=650M \ --innodb_log_buffer_size=16M \ --innodb_lock_wait_timeout=50 \ --innodb_file_io_threads=10 \ --innodb_flush_log_at_trx_commit=1 \ --innodb_flush_method=O_DIRECT \ --innodb_log_files_in_group=6 \ --innodb_support_xa=0 \ --innodb_doublewrite=0 \ --innodb-status-file=0 \ --skip-innodb-adaptive-hash-index \ --innodb_max_dirty_pages_pct=70 \ --innodb_thread_concurrency=0" [1] caneland.mysql.com Fedora release 10 Quad HexaCore @ 2.9GHz. Intel Xeon with 16MB L2 cache, 32GB RAM, hardware RAID 10 on 4 x 15k RPM SAS disks. [2] lu0009.mysql.com Ubuntu 8.10 Quad DualCore @ 3.4Ghz. Intel Xeon (with hyperthreading) with 16MB L2 cache, 8GB RAM, hardware RAID 10 on 4 x 15k RPM SAS disks. This machine has unofficial Intel CPUs.  [3] nehalem-1.mysql.com Ubuntu 8.10 Dual DualCore @ 2.4Ghz. Intel Xeon (with SMT) with 16MB L2 cache, 16GB RAM, RAID10 on 4 x SATA disk. [4] walldorf@home RHEL 5 Dual DualCore @ 2Ghz. Intel Xeon with 4MB L2 cache, 8GB RAM, hardware RAID 10 on 4 x 15k RPM SAS disks. Best regards, Hakan -- Hakan Küçükyılmaz, Senior Software Engineer DBTG/MySQL +49 160 98953296 Sun Microsystems GmbH Sonnenallee 1, DE-85551 Kirchheim-Heimstetten Geschaeftsfuehrer: Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer Vorsitz d. Aufs.rat.: Martin Haering HRB MUC 161028 49.011, 8.376