List:General Discussion« Previous MessageNext Message »
From:Jeremy Zawodny Date:August 8 2003 11:18pm
Subject:Re: Corrupt index = mysqld freeze?
View as plain text  
On Thu, Aug 07, 2003 at 09:07:41PM -0500, Dan Nelson wrote:
> In the last episode (Aug 07), Jeremy Zawodny said:
> > > I've been on the box at the mysql prompt quite a few times when it
> > > has happened and there was always a large amount of threads waiting
> > > for a lock to clear, and as soon as they went through nothing could
> > > connect, but this doesn't happen everytime we have a large queue,
> > > so there must be something else in the mix.  If you think any info
> > > I have might help you, let me know.  I'd love to hear any ideas you
> > > have.
> > 
> > I don't know how to do this with pthreads but with LT, I'd like to
> > identify a few of the pids for the struck threads and then get a
> > snapshot of the call stack to see where they're waiting.
> 
> The pstack command (ports/sysutils/pstack) knows about FreeBSD's libc_r
> threads, so you can run that to get a snapshot of mysql's thread
> state.

Just caught one.  We have nealy 300 "stuck" connections and they all
look like this right now:

45855: /home/y/libexec/mysqld
----------------- thread 0 (running) -----------------
 683f4de8 _sigsuspend (8e0ffbc0, 4, 8e0ff9c4, 683644de, 68373e64, 8dfff9e8) + 8
 68364534 __pthread_suspend_old (8e0ffbc0, 0, 8e0ffa04, 6836183a, 68373e64, 3ce3500c) + 64
 683619b6 __pthread_alt_lock (8366b04, 0, 8e0ffa34, 68364c5a, 0, 3ce3500c) + 18a
 68364db2 pthread_mutex_lock (8366af4, 3ce35504, 0, 826a3b9, 3ce35504, 3ce3500c) + 166
 8078c01 ip_to_hostname__FP7in_addrPUi (3ce35504, 8e0ffa9c, 8e0ffb24, 807d490, 3ce1d280,
3ce35504) + f1
 807d4a0 check_connections__FP3THD (3ce35000, 0, 1, 807d7dc, 68373e64, 8e0ffbc0) + c4
 807d8b2 handle_one_connection (3ce35000, 2, 8e0ffc78, 68365e77, 68373e64, 8e0ffbb8) + 11a
 68365f2d _init (8e0ffbc0, 8e0ffbc0, 0, 0, 0, 0) + f119

Now, to figure out what that's telling us...

Jeremy
-- 
Jeremy D. Zawodny     |  Perl, Web, MySQL, Linux Magazine, Yahoo!
<Jeremy@stripped>  |  http://jeremy.zawodny.com/

MySQL 4.0.13: up 7 days, processed 250,137,018 queries (400/sec. avg)
Thread
Corrupt index = mysqld freeze?Jennifer Goodie7 Aug
  • Re: Corrupt index = mysqld freeze?Jeremy Zawodny7 Aug
    • RE: Corrupt index = mysqld freeze?Jennifer Goodie8 Aug
      • Re: Corrupt index = mysqld freeze?Jeremy Zawodny8 Aug
        • RE: Corrupt index = mysqld freeze?Jennifer Goodie8 Aug
        • Re: Corrupt index = mysqld freeze?Dan Nelson8 Aug
          • Re: Corrupt index = mysqld freeze?Jeremy Zawodny9 Aug
            • Re: Corrupt index = mysqld freeze?Jeremy Zawodny9 Aug
              • RE: Corrupt index = mysqld freeze?Jennifer Goodie14 Aug
                • Re: Corrupt index = mysqld freeze?Jeremy Zawodny18 Aug
  • Re: Corrupt index = mysqld freeze?Daniel Kasak8 Aug
  • Re: Corrupt index = mysqld freeze?Ask Bjørn Hansen11 Aug