List:Internals« Previous MessageNext Message »
From:Marc Alff Date:June 5 2009 3:05pm
Subject:Re: Profiling a storage engine
View as plain text  
Hi Slava

Slava Akhmechet wrote:
> Hi,
>
> We're at a point where we need to understand performance bottlenecks
> of our storage engine and we're not sure how to proceed. What is the
> best way to do profiling? We are running benchmarks that compare our
> performance, cpu utilization, and IO throughput to MyISAM. We're also
> using procsystime to give us a picture of how much time we spend on
> various system calls. However, we're not sure how to best make sense
> of the numbers. We'd like to run gprof, but we're not sure how to
> compile MySQL to support it, and whether it will give realistic
> numbers.
>
> We'd appreciate if anyone could share their experience on how to
> effectively profile a storage engine.
>   
It will depend on your version and your platform.

Since you use procsystime, I suppose you might be
on a platform where you can take advantage of the
DTrace hooking that's in MySQL 5.4. That's worth a
look because it's performance-oriented and will be main.

For anything that might run on Linux or Windows,
or requires monitoring with lower overhead than DTrace,
I expect that the future lies with Performance Schema.
I will be happy to answer questions about using this.
The rest of this email is simply a bunch of
pointers to publicly available blogs, source code,
and specifications.

Blog postings in order of appearance:
http://blogs.mysql.com/peterg/2009/02/05/mysql-performance-schema/
http://blogs.mysql.com/peterg/2009/02/06/mysql-performance-schema-2/
http://blogs.mysql.com/peterg/2009/02/09/mysql-performance-schema-3/
http://blogs.mysql.com/peterg/2009/02/10/mysql-performance-schema-4/
http://blogs.mysql.com/peterg/2009/02/11/mysql-performance-schema-5/
http://blogs.mysql.com/peterg/2009/02/12/mysql-performance-schema-6/
http://blogs.mysql.com/peterg/2009/02/13/todo-mysql-performance-schema-7/

Source code of server including Performance Schema:
https://code.launchpad.net/~marc.alff/mysql-server/mysql-6.0-perfschema

Specifications:
WL#2360: Performance Schema
 http://forge.mysql.com/worklog/task.php?id=2360
WL#2515: Performance statements
 http://forge.mysql.com/worklog/task.php?id=2515
WL#3249: SHOW PROCESSLIST should show memory
 http://forge.mysql.com/worklog/task.php?id=3249
WL#4674: PERFORMANCE_SCHEMA Setup For Actors
 http://forge.mysql.com/worklog/task.php?id=4674
WL#4678: PERFORMANCE_SCHEMA Instrumenting File IO
 http://forge.mysql.com/worklog/task.php?id=4678
WL#4813 PERFORMANCE_SCHEMA Instrumenting Stages
 http://forge.mysql.com/worklog/task.php?id=4813
WL#4816: PERFORMANCE_SCHEMA Summaries
 http://forge.mysql.com/worklog/task.php?id=4816
WL#4895: PERFORMANCE_SCHEMA Instrumenting Table IO
 http://forge.mysql.com/worklog/task.php?id=4895
WL#4878: PERFORMANCE_SCHEMA Trace
 http://forge.mysql.com/worklog/task.php?id=4878
WL#4896: PERFORMANCE_SCHEMA Instrumenting Net IO
 http://forge.mysql.com/worklog/task.php?id=4896
WL#2333: SHOW ENGINE ... LOCK STATUS
 http://forge.mysql.com/worklog/task.php?id=2333

Regards,
-- Marc


Thread
Profiling a storage engineSlava Akhmechet4 Jun
  • Re: Profiling a storage engineMARK CALLAGHAN4 Jun
    • Re: Profiling a storage engineSlava Akhmechet5 Jun
      • Re: Profiling a storage engineMARK CALLAGHAN5 Jun
    • Re: Profiling a storage engineMichael Widenius5 Jun
      • Re: Profiling a storage engineMARK CALLAGHAN5 Jun
        • Re: Profiling a storage engineMichael Widenius5 Jun
  • Re: Profiling a storage engineAntony Dovgal5 Jun
  • Re: Profiling a storage engineMarc Alff5 Jun
    • Re: Profiling a storage engineSlava Akhmechet8 Jun