Hey,
Late last night, I got a call that one of our servers was down. Turns out
the machine had all 2500 connection slots in use, but none of them appeared
to be doing anything: hardly any CPU was being used, load 0.05, and tcpdump
confirmed that pretty much no data was being sent, either.
Disks were writeable - so not hanging write flood - and when I eventually
sent a kill, the daemon did a clean shutdown and came back up without a
hitch.
Unfortunately I couldn't log in while this was going on because I never
configured a few extra connections for super - that's been fixed by now.
Munin shows no discernable cause, no ramp-up on the load or whatever on
either the DB or the webservers; just a very sudden increase in open
connections and an equally sudden drop in network traffic.
Neither slowlog nor sys/errorlog shows anything out of the ordinary.
I can think of only two causes: bug in the app code, or bug in MySQL. This
is 5.0.32-Debian_7etch3-log (Debian package) on Debian Etch, running on
x86_64.
Does anyone know of similar things happening ?
--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel
