On Fri, Jun 5, 2009 at 8:05 AM, Marc Alff<Marc.Alff@stripped> wrote:
>> We're at a point where we need to understand performance bottlenecks
>> of our storage engine and we're not sure how to proceed.
Thanks to everyone for replying. We ended up going with a solution
we're very happy with (for now).
We run unix sar utility while we run the benchmarks and collect us
much data about the CPU, page cache, and IO as we can. We then plot
the data and this gives us a very clear picture of what is happening
with the entire system, so we can easily tell if our code is IO/CPU
bound, how and when this changes, and how it depends on various
architectures we test on. We can also easily tell how our code
stresses the system compared to MyISAM and InnoDB, which gives us good
insights into where we go right and where we go wrong.
FYI, we run sar -bBru, which gives all the information we need. We
then run oprofile to precisely determine CPU bottlenecks for the CPU
bound parts of the code, and we know our code well enough to make good
guesses about where IO-bound code stalls (we'd like to guess less
here, and we might hack DBUG_ENTER/DBUG_RETURN to record time
information, but we have enough information for now).
- Slava Akhmechet