Mike,
I re-posted your Show Status to the group to see if anyone can offer
a way to speed up the indexing for you.
BTW, you are adding ALL of the indexes to the table using ONE sql statement
right? And not a separate SQL statement to build each index?
Mike
At 02:01 AM 6/25/2009, you wrote:
>Like I said in the subject line, I am using 5.1.34. I started with
>my-huge.cnf, which says "key_buffer" rather than "key_buffer_size"; SHOW
>GLOBAL VARIABLES confirms, however, that my key_buffer_size is 8GB.
>
>That indexing operation finally finished after about 1.5 hours; that was
>about 0.5 hours ago. Now I am on to other things. Here is the status you
>suggested:
>
>+-----------------------------------+-----------+
>| Variable_name | Value |
>+-----------------------------------+-----------+
>| Aborted_clients | 0 |
>| Aborted_connects | 0 |
>| Binlog_cache_disk_use | 0 |
>| Binlog_cache_use | 0 |
>| Bytes_received | 135 |
>| Bytes_sent | 1405 |
>| Com_admin_commands | 0 |
>| Com_assign_to_keycache | 0 |
>| Com_alter_db | 0 |
>| Com_alter_db_upgrade | 0 |
>| Com_alter_event | 0 |
>| Com_alter_function | 0 |
>| Com_alter_procedure | 0 |
>| Com_alter_server | 0 |
>| Com_alter_table | 0 |
>| Com_alter_tablespace | 0 |
>| Com_analyze | 0 |
>| Com_backup_table | 0 |
>| Com_begin | 0 |
>| Com_binlog | 0 |
>| Com_call_procedure | 0 |
>| Com_change_db | 0 |
>| Com_change_master | 0 |
>| Com_check | 0 |
>| Com_checksum | 0 |
>| Com_commit | 0 |
>| Com_create_db | 0 |
>| Com_create_event | 0 |
>| Com_create_function | 0 |
>| Com_create_index | 0 |
>| Com_create_procedure | 0 |
>| Com_create_server | 0 |
>| Com_create_table | 0 |
>| Com_create_trigger | 0 |
>| Com_create_udf | 0 |
>| Com_create_user | 0 |
>| Com_create_view | 0 |
>| Com_dealloc_sql | 0 |
>| Com_delete | 0 |
>| Com_delete_multi | 0 |
>| Com_do | 0 |
>| Com_drop_db | 0 |
>| Com_drop_event | 0 |
>| Com_drop_function | 0 |
>| Com_drop_index | 0 |
>| Com_drop_procedure | 0 |
>| Com_drop_server | 0 |
>| Com_drop_table | 0 |
>| Com_drop_trigger | 0 |
>| Com_drop_user | 0 |
>| Com_drop_view | 0 |
>| Com_empty_query | 0 |
>| Com_execute_sql | 0 |
>| Com_flush | 0 |
>| Com_grant | 0 |
>| Com_ha_close | 0 |
>| Com_ha_open | 0 |
>| Com_ha_read | 0 |
>| Com_help | 0 |
>| Com_insert | 0 |
>| Com_insert_select | 0 |
>| Com_install_plugin | 0 |
>| Com_kill | 0 |
>| Com_load | 0 |
>| Com_load_master_data | 0 |
>| Com_load_master_table | 0 |
>| Com_lock_tables | 0 |
>| Com_optimize | 0 |
>| Com_preload_keys | 0 |
>| Com_prepare_sql | 0 |
>| Com_purge | 0 |
>| Com_purge_before_date | 0 |
>| Com_release_savepoint | 0 |
>| Com_rename_table | 0 |
>| Com_rename_user | 0 |
>| Com_repair | 0 |
>| Com_replace | 0 |
>| Com_replace_select | 0 |
>| Com_reset | 0 |
>| Com_restore_table | 0 |
>| Com_revoke | 0 |
>| Com_revoke_all | 0 |
>| Com_rollback | 0 |
>| Com_rollback_to_savepoint | 0 |
>| Com_savepoint | 0 |
>| Com_select | 1 |
>| Com_set_option | 0 |
>| Com_show_authors | 0 |
>| Com_show_binlog_events | 0 |
>| Com_show_binlogs | 0 |
>| Com_show_charsets | 0 |
>| Com_show_collations | 0 |
>| Com_show_column_types | 0 |
>| Com_show_contributors | 0 |
>| Com_show_create_db | 0 |
>| Com_show_create_event | 0 |
>| Com_show_create_func | 0 |
>| Com_show_create_proc | 0 |
>| Com_show_create_table | 0 |
>| Com_show_create_trigger | 0 |
>| Com_show_databases | 0 |
>| Com_show_engine_logs | 0 |
>| Com_show_engine_mutex | 0 |
>| Com_show_engine_status | 0 |
>| Com_show_events | 0 |
>| Com_show_errors | 0 |
>| Com_show_fields | 0 |
>| Com_show_function_status | 0 |
>| Com_show_grants | 0 |
>| Com_show_keys | 0 |
>| Com_show_master_status | 0 |
>| Com_show_new_master | 0 |
>| Com_show_open_tables | 0 |
>| Com_show_plugins | 0 |
>| Com_show_privileges | 0 |
>| Com_show_procedure_status | 0 |
>| Com_show_processlist | 0 |
>| Com_show_profile | 0 |
>| Com_show_profiles | 0 |
>| Com_show_slave_hosts | 0 |
>| Com_show_slave_status | 0 |
>| Com_show_status | 1 |
>| Com_show_storage_engines | 0 |
>| Com_show_table_status | 0 |
>| Com_show_tables | 0 |
>| Com_show_triggers | 0 |
>| Com_show_variables | 1 |
>| Com_show_warnings | 0 |
>| Com_slave_start | 0 |
>| Com_slave_stop | 0 |
>| Com_stmt_close | 0 |
>| Com_stmt_execute | 0 |
>| Com_stmt_fetch | 0 |
>| Com_stmt_prepare | 0 |
>| Com_stmt_reprepare | 0 |
>| Com_stmt_reset | 0 |
>| Com_stmt_send_long_data | 0 |
>| Com_truncate | 0 |
>| Com_uninstall_plugin | 0 |
>| Com_unlock_tables | 0 |
>| Com_update | 0 |
>| Com_update_multi | 0 |
>| Com_xa_commit | 0 |
>| Com_xa_end | 0 |
>| Com_xa_prepare | 0 |
>| Com_xa_recover | 0 |
>| Com_xa_rollback | 0 |
>| Com_xa_start | 0 |
>| Compression | OFF |
>| Connections | 9 |
>| Created_tmp_disk_tables | 0 |
>| Created_tmp_files | 13 |
>| Created_tmp_tables | 1 |
>| Delayed_errors | 0 |
>| Delayed_insert_threads | 0 |
>| Delayed_writes | 0 |
>| Flush_commands | 1 |
>| Handler_commit | 0 |
>| Handler_delete | 0 |
>| Handler_discover | 0 |
>| Handler_prepare | 0 |
>| Handler_read_first | 0 |
>| Handler_read_key | 0 |
>| Handler_read_next | 0 |
>| Handler_read_prev | 0 |
>| Handler_read_rnd | 0 |
>| Handler_read_rnd_next | 34 |
>| Handler_rollback | 0 |
>| Handler_savepoint | 0 |
>| Handler_savepoint_rollback | 0 |
>| Handler_update | 0 |
>| Handler_write | 33 |
>| Innodb_buffer_pool_pages_data | 19 |
>| Innodb_buffer_pool_pages_dirty | 0 |
>| Innodb_buffer_pool_pages_flushed | 0 |
>| Innodb_buffer_pool_pages_free | 493 |
>| Innodb_buffer_pool_pages_misc | 0 |
>| Innodb_buffer_pool_pages_total | 512 |
>| Innodb_buffer_pool_read_ahead_rnd | 1 |
>| Innodb_buffer_pool_read_ahead_seq | 0 |
>| Innodb_buffer_pool_read_requests | 77 |
>| Innodb_buffer_pool_reads | 12 |
>| Innodb_buffer_pool_wait_free | 0 |
>| Innodb_buffer_pool_write_requests | 0 |
>| Innodb_data_fsyncs | 3 |
>| Innodb_data_pending_fsyncs | 0 |
>| Innodb_data_pending_reads | 0 |
>| Innodb_data_pending_writes | 0 |
>| Innodb_data_read | 2494464 |
>| Innodb_data_reads | 25 |
>| Innodb_data_writes | 3 |
>| Innodb_data_written | 1536 |
>| Innodb_dblwr_pages_written | 0 |
>| Innodb_dblwr_writes | 0 |
>| Innodb_log_waits | 0 |
>| Innodb_log_write_requests | 0 |
>| Innodb_log_writes | 1 |
>| Innodb_os_log_fsyncs | 3 |
>| Innodb_os_log_pending_fsyncs | 0 |
>| Innodb_os_log_pending_writes | 0 |
>| Innodb_os_log_written | 512 |
>| Innodb_page_size | 16384 |
>| Innodb_pages_created | 0 |
>| Innodb_pages_read | 19 |
>| Innodb_pages_written | 0 |
>| Innodb_row_lock_current_waits | 0 |
>| Innodb_row_lock_time | 0 |
>| Innodb_row_lock_time_avg | 0 |
>| Innodb_row_lock_time_max | 0 |
>| Innodb_row_lock_waits | 0 |
>| Innodb_rows_deleted | 0 |
>| Innodb_rows_inserted | 0 |
>| Innodb_rows_read | 0 |
>| Innodb_rows_updated | 0 |
>| Key_blocks_not_flushed | 26 |
>| Key_blocks_unused | 6844469 |
>| Key_blocks_used | 676158 |
>| Key_read_requests | 635624989 |
>| Key_reads | 12664 |
>| Key_write_requests | 112359740 |
>| Key_writes | 689890 |
>| Last_query_cost | 10.499000 |
>| Max_used_connections | 2 |
>| Not_flushed_delayed_rows | 0 |
>| Open_files | 70 |
>| Open_streams | 0 |
>| Open_table_definitions | 38 |
>| Open_tables | 37 |
>| Opened_files | 312 |
>| Opened_table_definitions | 0 |
>| Opened_tables | 0 |
>| Prepared_stmt_count | 0 |
>| Qcache_free_blocks | 1 |
>| Qcache_free_memory | 33536880 |
>| Qcache_hits | 0 |
>| Qcache_inserts | 0 |
>| Qcache_lowmem_prunes | 0 |
>| Qcache_not_cached | 12 |
>| Qcache_queries_in_cache | 0 |
>| Qcache_total_blocks | 1 |
>| Queries | 98 |
>| Questions | 3 |
>| Rpl_status | NULL |
>| Select_full_join | 0 |
>| Select_full_range_join | 0 |
>| Select_range | 0 |
>| Select_range_check | 0 |
>| Select_scan | 1 |
>| Slave_open_temp_tables | 0 |
>| Slave_retried_transactions | 0 |
>| Slave_running | OFF |
>| Slow_launch_threads | 0 |
>| Slow_queries | 0 |
>| Sort_merge_passes | 0 |
>| Sort_range | 0 |
>| Sort_rows | 0 |
>| Sort_scan | 0 |
>| Ssl_accept_renegotiates | 0 |
>| Ssl_accepts | 0 |
>| Ssl_callback_cache_hits | 0 |
>| Ssl_cipher | |
>| Ssl_cipher_list | |
>| Ssl_client_connects | 0 |
>| Ssl_connect_renegotiates | 0 |
>| Ssl_ctx_verify_depth | 0 |
>| Ssl_ctx_verify_mode | 0 |
>| Ssl_default_timeout | 0 |
>| Ssl_finished_accepts | 0 |
>| Ssl_finished_connects | 0 |
>| Ssl_session_cache_hits | 0 |
>| Ssl_session_cache_misses | 0 |
>| Ssl_session_cache_mode | NONE |
>| Ssl_session_cache_overflows | 0 |
>| Ssl_session_cache_size | 0 |
>| Ssl_session_cache_timeouts | 0 |
>| Ssl_sessions_reused | 0 |
>| Ssl_used_session_cache_entries | 0 |
>| Ssl_verify_depth | 0 |
>| Ssl_verify_mode | 0 |
>| Ssl_version | |
>| Table_locks_immediate | 101 |
>| Table_locks_waited | 0 |
>| Tc_log_max_pages_used | 0 |
>| Tc_log_page_size | 0 |
>| Tc_log_page_waits | 0 |
>| Threads_cached | 0 |
>| Threads_connected | 2 |
>| Threads_created | 2 |
>| Threads_running | 2 |
>| Uptime | 6952 |
>| Uptime_since_flush_status | 6952 |
>+-----------------------------------+-----------+
>
>Thanks,
>Mike Spreitzer
>
>
>
>mos <mos99@stripped>
>
>06/25/09 02:32 AM
>To
>mysql@stripped
>cc
>Subject
>Re: Indexing dynamics in MySQL Community Edition 5.1.34
>
>
>
>
>At 12:37 AM 6/25/2009, you wrote:
> >Actually, my characterization of the current state is wrong. It appears
> >that one core is completely busy, I suppose MySQL does this indexing work
> >in a single thread. Is it reasonable for indexing to be CPU bound?
> >
> >
> >my.cnf based on my-huge.cnf, expanding key_buffer to 8G,
> >myisam_sort_buffer_size to 256M, and putting tmpdir on the fiber channel
> >disk.
>
>Mike,
> You mean "key_buffer_size" don't you and not "key_buffer"? If you are
>using MySQL 5.0 then the upper limit is 4gb but 5.1 allows for more than 4gb.
>Also try increasing "sort_buffer_size".
>
>Posting your "Show Status" will help people see where the bottle neck is.
>
>Mike
>
>
>--
>MySQL General Mailing List
>For list archives: http://lists.mysql.com/mysql
>To unsubscribe: http://lists.mysql.com/mysql?unsub=1
>
