On Fri, Aug 08, 2003 at 04:18:51PM -0700, Jeremy Zawodny wrote:
> On Thu, Aug 07, 2003 at 09:07:41PM -0500, Dan Nelson wrote:
> > In the last episode (Aug 07), Jeremy Zawodny said:
> > > > I've been on the box at the mysql prompt quite a few times when it
> > > > has happened and there was always a large amount of threads waiting
> > > > for a lock to clear, and as soon as they went through nothing could
> > > > connect, but this doesn't happen everytime we have a large queue,
> > > > so there must be something else in the mix. If you think any info
> > > > I have might help you, let me know. I'd love to hear any ideas you
> > > > have.
> > >
> > > I don't know how to do this with pthreads but with LT, I'd like to
> > > identify a few of the pids for the struck threads and then get a
> > > snapshot of the call stack to see where they're waiting.
> > The pstack command (ports/sysutils/pstack) knows about FreeBSD's libc_r
> > threads, so you can run that to get a snapshot of mysql's thread
> > state.
> Just caught one. We have nealy 300 "stuck" connections and they all
> look like this right now:
> 45855: /home/y/libexec/mysqld
> ----------------- thread 0 (running) -----------------
> 683f4de8 _sigsuspend (8e0ffbc0, 4, 8e0ff9c4, 683644de, 68373e64, 8dfff9e8) + 8
> 68364534 __pthread_suspend_old (8e0ffbc0, 0, 8e0ffa04, 6836183a, 68373e64, 3ce3500c)
> + 64
> 683619b6 __pthread_alt_lock (8366b04, 0, 8e0ffa34, 68364c5a, 0, 3ce3500c) + 18a
> 68364db2 pthread_mutex_lock (8366af4, 3ce35504, 0, 826a3b9, 3ce35504, 3ce3500c) +
> 8078c01 ip_to_hostname__FP7in_addrPUi (3ce35504, 8e0ffa9c, 8e0ffb24, 807d490,
> 3ce1d280, 3ce35504) + f1
> 807d4a0 check_connections__FP3THD (3ce35000, 0, 1, 807d7dc, 68373e64, 8e0ffbc0) +
> 807d8b2 handle_one_connection (3ce35000, 2, 8e0ffc78, 68365e77, 68373e64, 8e0ffbb8)
> + 11a
> 68365f2d _init (8e0ffbc0, 8e0ffbc0, 0, 0, 0, 0) + f119
> Now, to figure out what that's telling us...
Setting skip-name-resolve will avoid this code path and the bug. It
looks like some DNS funkyness...
Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo!
<Jeremy@stripped> | http://jeremy.zawodny.com/
MySQL 4.0.13: up 7 days, processed 250,298,423 queries (400/sec. avg)