List:General Discussion« Previous MessageNext Message »
From:Imran Chaudhry Date:April 20 2010 11:02am
Subject:Analysis of a weeks worth of general log
View as plain text  
I have 7 days worth of general log data totalling 4.4GB.

I want to analyze this data to get:

a) queries per second, minute, hour and day
b) a count of the number of selects versus write statements (delete,
insert, replace and update)
c) a variation of the above with "select, replace, delete and insert"
versus "update"

How can I do this?

I've looked at mysqlsla which is complex, works well but does not
quite get what I want. [1]

I looked at MyProfi 0.18 which looks like it will get some of the
answers but runs out of memory working on the smallest log file
(mysql.log) even with memory_limit in php.ini set to 1024MB [2]

-rw-r----- 1 imran imran 268M 2010-04-19 13:03 mysql.log
-rw-r----- 1 imran imran 721M 2010-04-19 12:56 mysql.log.1
-rw-r----- 1 imran imran 737M 2010-04-19 13:05 mysql.log.2
-rw-r----- 1 imran imran 554M 2010-04-19 13:06 mysql.log.3
-rw-r----- 1 imran imran 499M 2010-04-19 13:02 mysql.log.4
-rw-r----- 1 imran imran 568M 2010-04-19 12:59 mysql.log.5
-rw-r----- 1 imran imran 488M 2010-04-19 13:01 mysql.log.6

Any pointers please? If all else fails, I will prolly write a perl
script to munge it.

[1] http://hackmysql.com/mysqlsla
[2] http://myprofi.sourceforge.net

-- 
GPG Key fingerprint = B323 477E F6AB 4181 9C65  F637 BC5F 7FCC 9CC9 CC7F
Thread
Analysis of a weeks worth of general logImran Chaudhry20 Apr
  • Re: Analysis of a weeks worth of general logJohan De Meersman20 Apr
  • Re: Analysis of a weeks worth of general logJim Lyons20 Apr
    • Re: Analysis of a weeks worth of general logAnand Kumar20 Apr
    • Re: Analysis of a weeks worth of general logCarsten Pedersen20 Apr
      • Re: Analysis of a weeks worth of general logCarsten Pedersen20 Apr
        • Re: Analysis of a weeks worth of general logJohan De Meersman20 Apr
        • Re: Analysis of a weeks worth of general logImran Chaudhry20 Apr