List:General Discussion« Previous MessageNext Message »
From:Reindl Harald Date:April 13 2012 2:35pm
Subject:Re: FYI: how to prevent mysql from oom-killer
View as plain text  
hm - my servers generally have no swap or better said
only a 500 MB swap-file, they are all virtual machines
in a ESXi cluster

you really do not want a VM swapping
this leads in unusaeable performance

in my case the problem was another process eating up all memory
by user-mistake and my hint is to make sure in this case that
this process get killed and never mysqld

but i never ever had to reboot any linux guest to get it back to life

Am 13.04.2012 16:27, schrieb Johnny Withers:
> I had this same problem with the OOM for a while. Very frustrating to have to reboot
> a server to bring it back to life.
> 
> I found out the OOM only ran when the swap file was about 99% full. The servers I had
> this problem on had 16GB and
> 24GB of ram, but only 2GB of swap. I increased the swap on the 24GB servers to 48GB
> and on the 16GB servers to
> 32GB. The swap never fills to over 60% now and I haven't' had any OOM problems since
> and the systems run great.
> 
> I've also set vm.swappiness=0 in /etc/sysctl.conf
> 
> -JW
> 
> On Fri, Apr 13, 2012 at 7:17 AM, Reindl Harald <h.reindl@stripped
> <mailto:h.reindl@stripped>> wrote:
> 
>     the following may be useful for most server systems
> 
>     OOM-killer acts if some process reclaims more and more
>     memory and the kernel randomly kills unimportant tasks
>     using hughe memory
> 
>     in case of a running mysqld the classification "unimportant"
>     is nearly all time wrong and can cause hughe damage and work
>     in other words: you really never want killed a database server
>     randomly instead "dbmail-imapd" which can be restarted via
>     systemd without pain and may be the root-cause of OOM
>     _________________________________________
> 
>     with one single command you can protect processes from get killed
>     i started to run this every 15 minutes to make sure it is also
>     active after restarts
> 
>     i am considering include this in "mysqld.service" as
>     "ExecStartPost=-/usr/local/bin/mysql-no-oom.sh" in our
>     internal mysqld-packages and include also the script
>     _________________________________________
> 
>     [root@mail:~]$ cat /etc/crontab | grep oom
>     0,15,30,45 * * * * root bash /usr/local/bin/mysql-no-oom.sh
> 
>     [root@mail:~]$ cat /usr/local/bin/mysql-no-oom.sh
>     #!/bin/bash
>     pgrep -f "/usr/libexec/mysqld" | while read PID; do echo -1000 >
> /proc/$PID/oom_score_adj; done
> 
> 
> 
> 
> -- 
> -----------------------------
> Johnny Withers
> 601.209.4985
> johnny@stripped <mailto:johnny@stripped>


Attachment: [application/pgp-signature] OpenPGP digital signature signature.asc
Thread
FYI: how to prevent mysql from oom-killerReindl Harald13 Apr
  • Re: FYI: how to prevent mysql from oom-killerMihamina Rakotomandimby13 Apr
    • Re: FYI: how to prevent mysql from oom-killerReindl Harald13 Apr
  • Re: FYI: how to prevent mysql from oom-killerJohnny Withers13 Apr
    • Re: FYI: how to prevent mysql from oom-killerReindl Harald13 Apr