List:General Discussion« Previous MessageNext Message »
From:prabhat kumar Date:December 23 2009 8:45pm
Subject:Re: HELP! "RESET MASTER" hosed replication
View as plain text  
*Next time look for 'purge binary logs'!*

Purge may also cause problem , before doing purge make sure slave is in sync
with master or at least cross check slave currently which bin file getting
sync.

On Thu, Dec 24, 2009 at 2:05 AM, Carlos Proal <carlos.proal@stripped>wrote:

>
> The issue is that replication relies on  this logs !!!, so when you deleted
> them .....
> Generally speaking you have to:
> > stop the slave
> > sync the master with the slave (there are several ways to do this and
> depending how busy is your master)
> >grab the master status (position)
> >change the slave to point to the master new position
> >start the slave
>
> Hope this helps.
>
> Carlos
>
>
> On 12/23/2009 2:20 PM, Daevid Vincent wrote:
>
>> I got an alert that one of the drives was filling up (3% free). So I
>> figured out that a large chunk was from /var/log/mysql
>>
>> root@pse10:~# find / -type d -print0 | xargs -0 -n1 du -sk | sort -rn |
>> head -n20>  ~/dir-sizes.txt
>>
>> root@pse10:~# cat ~/dir-sizes.txt
>> 159121012 /
>> 70442396  /var
>> 70127764  /var/log
>> 69991160  /var/log/mysql<-------- big offender
>> 56307436  /data
>> 31479936  /home
>> 29386076  /data/mysql
>> 26899784  /data/archive
>>
>> It looked like the /var/log/mysql was pretty full of these
>> Mysql-bin.00XXXX
>> log files,
>>
>> ...
>> -rw-rw---- 1 mysql adm 105019928 2009-12-23 05:07 mysql-bin.001196
>> -rw-rw---- 1 mysql adm 105004751 2009-12-23 05:08 mysql-bin.001197
>> -rw-rw---- 1 mysql adm 104978518 2009-12-23 05:10 mysql-bin.001198
>> -rw-rw---- 1 mysql adm 104949073 2009-12-23 05:11 mysql-bin.001199
>> -rw-rw---- 1 mysql adm 104925795 2009-12-23 05:13 mysql-bin.001200
>> -rw-rw---- 1 mysql adm 104974354 2009-12-23 05:14 mysql-bin.001201
>> -rw-rw---- 1 mysql adm 105089249 2009-12-23 05:16 mysql-bin.001202
>> -rw-rw---- 1 mysql adm 105165487 2009-12-23 05:17 mysql-bin.001203
>> -rw-rw---- 1 mysql adm 104926853 2009-12-23 05:19 mysql-bin.001204
>> -rw-rw---- 1 mysql adm 105139076 2009-12-23 05:20 mysql-bin.001205
>> -rw-rw---- 1 mysql adm 104891552 2009-12-23 05:22 mysql-bin.001206
>> -rw-rw---- 1 mysql adm 104959626 2009-12-23 05:25 mysql-bin.001207
>> -rw-rw---- 1 mysql adm 104883048 2009-12-23 05:27 mysql-bin.001208
>> -rw-rw---- 1 mysql adm 104993511 2009-12-23 05:28 mysql-bin.001209
>> -rw-rw---- 1 mysql adm 104945974 2009-12-23 05:30 mysql-bin.001210
>> -rw-rw---- 1 mysql adm  35468892 2009-12-23 05:30 mysql-bin.001211
>> -rw-rw---- 1 mysql adm     21728 2009-12-23 05:30 mysql-bin.index
>> -rw-r----- 1 mysql adm     12836 2009-12-23 00:12 mysql-slow.log
>> ...
>>
>> so I took the liberty of resetting them...
>> http://dev.mysql.com/doc/refman/5.0/en/reset.html
>>
>> vincentd@pse10 /var/log/mysql $ dbroot
>> (root@localhost) [(none)]>  RESET MASTER;
>>
>> vincentd@pse10 /var/log/mysql $ ll
>> total 2792
>> -rw-rw---- 1 mysql adm 2801618 2009-12-23 05:35 mysql-bin.000001
>> -rw-rw---- 1 mysql adm      32 2009-12-23 05:35 mysql-bin.index
>> -rw-r----- 1 mysql adm   14987 2009-12-23 05:35 mysql-slow.log
>> -rw-r----- 1 mysql adm    1102 2009-12-22 00:13 mysql-slow.log.1.gz
>> -rw-r----- 1 mysql adm     891 2009-12-21 00:02 mysql-slow.log.2.gz
>> -rw-r----- 1 mysql adm    1318 2009-12-20 00:02 mysql-slow.log.3.gz
>> -rw-r----- 1 mysql adm     687 2009-12-19 00:02 mysql-slow.log.4.gz
>> -rw-r----- 1 mysql adm    5246 2009-12-17 20:38 mysql-slow.log.5.gz
>> -rw-r----- 1 mysql adm     156 2009-12-16 06:25 mysql-slow.log.6.gz
>> -rw-r----- 1 mysql adm    1114 2009-12-15 16:26 mysql-slow.log.7.gz
>>
>> Which freed up a tremendous amount of space again...
>>
>> However, a co-worker informed me that now our slaves are broken and
>> replication is hosed!
>> What did I do wrong or forget to do?
>> I see no mention of something I was supposed to do for replication
>> scenarios...
>> Was I supposed to "RESET SLAVE" too?
>>
>> As of right now, /var/log/mysql has grown to mysql-bin.000028 since last
>> night when I reset it.
>> How do I recover from this?
>>
>>
>>
>>
>
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe:
> http://lists.mysql.com/mysql?unsub=1
>
>


-- 
Best Regards,

Prabhat Kumar
MySQL DBA
Datavail-India Mumbai
Mobile     : 91-9987681929
www.datavail.com

My Blog: http://adminlinux.blogspot.com
My LinkedIn: http://www.linkedin.com/in/profileprabhat

Thread
HELP! "RESET MASTER" hosed replicationDaevid Vincent23 Dec
  • Re: HELP! "RESET MASTER" hosed replicationClaudio Nanni23 Dec
  • Re: HELP! "RESET MASTER" hosed replicationCarlos Proal23 Dec
    • Re: HELP! "RESET MASTER" hosed replicationClaudio Nanni23 Dec
      • Re: HELP! "RESET MASTER" hosed replicationCarlos Proal23 Dec
    • Re: HELP! "RESET MASTER" hosed replicationprabhat kumar23 Dec
      • RE: HELP! "RESET MASTER" hosed replicationDaevid Vincent23 Dec