List:General Discussion« Previous MessageNext Message »
From:Carsten Pedersen Date:April 20 2010 3:30pm
Subject:Re: Analysis of a weeks worth of general log
View as plain text  
Jim Lyons skrev:
> Has anyone tried using the log_output option in mysql 5.1 to have the
> general log put into a table and not a flat file?  I used it for a while
> before having to downgrade back to 5.0 but thought it was a great idea.  I'm
> curious to see if anyone feels it helps analysis.

I tried that once, and ran into some problems. Depending on your exact 
version, you might experience the same.

http://www.bitbybit.dk/carsten/blog/?p=115

(also has a number of good comments on analysis tools)

And yes, having the data available in a table is a Good Thing for analysis.

/ Carsten


> 
> On Tue, Apr 20, 2010 at 6:02 AM, Imran Chaudhry <ichaudhry@stripped> wrote:
> 
>> I have 7 days worth of general log data totalling 4.4GB.
>>
>> I want to analyze this data to get:
>>
>> a) queries per second, minute, hour and day
>> b) a count of the number of selects versus write statements (delete,
>> insert, replace and update)
>> c) a variation of the above with "select, replace, delete and insert"
>> versus "update"
>>
>> How can I do this?
>>
>> I've looked at mysqlsla which is complex, works well but does not
>> quite get what I want. [1]
>>
>> I looked at MyProfi 0.18 which looks like it will get some of the
>> answers but runs out of memory working on the smallest log file
>> (mysql.log) even with memory_limit in php.ini set to 1024MB [2]
>>
>> -rw-r----- 1 imran imran 268M 2010-04-19 13:03 mysql.log
>> -rw-r----- 1 imran imran 721M 2010-04-19 12:56 mysql.log.1
>> -rw-r----- 1 imran imran 737M 2010-04-19 13:05 mysql.log.2
>> -rw-r----- 1 imran imran 554M 2010-04-19 13:06 mysql.log.3
>> -rw-r----- 1 imran imran 499M 2010-04-19 13:02 mysql.log.4
>> -rw-r----- 1 imran imran 568M 2010-04-19 12:59 mysql.log.5
>> -rw-r----- 1 imran imran 488M 2010-04-19 13:01 mysql.log.6
>>
>> Any pointers please? If all else fails, I will prolly write a perl
>> script to munge it.
>>
>> [1] http://hackmysql.com/mysqlsla
>> [2] http://myprofi.sourceforge.net
>>
>> --
>> GPG Key fingerprint = B323 477E F6AB 4181 9C65  F637 BC5F 7FCC 9CC9 CC7F
>>
>> --
>> MySQL General Mailing List
>> For list archives: http://lists.mysql.com/mysql
>> To unsubscribe:    http://lists.mysql.com/mysql?unsub=1
>>
>>
> 
> 
Thread
Analysis of a weeks worth of general logImran Chaudhry20 Apr
  • Re: Analysis of a weeks worth of general logJohan De Meersman20 Apr
  • Re: Analysis of a weeks worth of general logJim Lyons20 Apr
    • Re: Analysis of a weeks worth of general logAnand Kumar20 Apr
    • Re: Analysis of a weeks worth of general logCarsten Pedersen20 Apr
      • Re: Analysis of a weeks worth of general logCarsten Pedersen20 Apr
        • Re: Analysis of a weeks worth of general logJohan De Meersman20 Apr
        • Re: Analysis of a weeks worth of general logImran Chaudhry20 Apr