The impact of swap activity on performance is dependent on the rate at
which things are being swapped and the speed of swapping. A few pages
per second probably won't kill things but in this case it was swapping
hundreds of pages per second which killed performance. Disks are much
slower than ram.
In my environment mysqld_safe will failover the cluster if it detects
mysqld has crashed so I prefer the quick crash and failover to the
long period of slow response time. Many operators prefer the long slow
response time, I don't but it's a religious debate.
On Sun, Apr 18, 2010 at 12:13 PM, Rob Wultsch <wultsch@stripped> wrote:
> On Sun, Apr 18, 2010 at 12:04 PM, Eric Bergen <eric.bergen@stripped> wrote:
>> Linux will normally swap out a few pages of rarely used memory so it's
>> a good idea to have some swap around. 2G seems excessive though.
>> Usually I prefer to have linux kill processes rather than excessively
>> swapping. I've worked on machines before that have swapped so badly
>> that it took minutes just to ssh to them. This is effectively a
>> failure scenario that can last for a lot longer than it takes to
>> restart/failover mysqld. For apache it means the clients will see
>> errors until the load balancer health check drops the server out of
>> rotation. The best solution in all cases is to keep an eye on swap
>> in/out and memory usage so neither the crash nor the excessive
>> swapping becomes a problem.
> Umm, you were probably horribly over io utilized. Swapping by itself
> will not kill perforance I have some boxes where mysql has leaked a
> metric crap ton of memory and swapping is ok. The leaked memory is
> swapped out and sits out in swap. Every now and a again I create more
> swap to keep the server happy.
> Swapping is often preferable to crash with unplanned downtime.
> Note that innodb_flush_method can implact this...
> Rob Wultsch