List:General Discussion« Previous MessageNext Message »
From:Jeremy Zawodny Date:August 8 2003 11:30pm
Subject:Re: Corrupt index = mysqld freeze?
View as plain text  
On Fri, Aug 08, 2003 at 04:18:51PM -0700, Jeremy Zawodny wrote:
> On Thu, Aug 07, 2003 at 09:07:41PM -0500, Dan Nelson wrote:
> > In the last episode (Aug 07), Jeremy Zawodny said:
> > > > I've been on the box at the mysql prompt quite a few times when it
> > > > has happened and there was always a large amount of threads waiting
> > > > for a lock to clear, and as soon as they went through nothing could
> > > > connect, but this doesn't happen everytime we have a large queue,
> > > > so there must be something else in the mix.  If you think any info
> > > > I have might help you, let me know.  I'd love to hear any ideas you
> > > > have.
> > > 
> > > I don't know how to do this with pthreads but with LT, I'd like to
> > > identify a few of the pids for the struck threads and then get a
> > > snapshot of the call stack to see where they're waiting.
> > 
> > The pstack command (ports/sysutils/pstack) knows about FreeBSD's libc_r
> > threads, so you can run that to get a snapshot of mysql's thread
> > state.
> 
> Just caught one.  We have nealy 300 "stuck" connections and they all
> look like this right now:
> 
> 45855: /home/y/libexec/mysqld
> ----------------- thread 0 (running) -----------------
>  683f4de8 _sigsuspend (8e0ffbc0, 4, 8e0ff9c4, 683644de, 68373e64, 8dfff9e8) + 8
>  68364534 __pthread_suspend_old (8e0ffbc0, 0, 8e0ffa04, 6836183a, 68373e64, 3ce3500c)
> + 64
>  683619b6 __pthread_alt_lock (8366b04, 0, 8e0ffa34, 68364c5a, 0, 3ce3500c) + 18a
>  68364db2 pthread_mutex_lock (8366af4, 3ce35504, 0, 826a3b9, 3ce35504, 3ce3500c) +
> 166
>  8078c01 ip_to_hostname__FP7in_addrPUi (3ce35504, 8e0ffa9c, 8e0ffb24, 807d490,
> 3ce1d280, 3ce35504) + f1
>  807d4a0 check_connections__FP3THD (3ce35000, 0, 1, 807d7dc, 68373e64, 8e0ffbc0) +
> c4
>  807d8b2 handle_one_connection (3ce35000, 2, 8e0ffc78, 68365e77, 68373e64, 8e0ffbb8)
> + 11a
>  68365f2d _init (8e0ffbc0, 8e0ffbc0, 0, 0, 0, 0) + f119
> 
> Now, to figure out what that's telling us...

Setting skip-name-resolve will avoid this code path and the bug.  It
looks like some DNS funkyness...

Jeremy
-- 
Jeremy D. Zawodny     |  Perl, Web, MySQL, Linux Magazine, Yahoo!
<Jeremy@stripped>  |  http://jeremy.zawodny.com/

MySQL 4.0.13: up 7 days, processed 250,298,423 queries (400/sec. avg)
Thread
Corrupt index = mysqld freeze?Jennifer Goodie7 Aug
  • Re: Corrupt index = mysqld freeze?Jeremy Zawodny7 Aug
    • RE: Corrupt index = mysqld freeze?Jennifer Goodie8 Aug
      • Re: Corrupt index = mysqld freeze?Jeremy Zawodny8 Aug
        • RE: Corrupt index = mysqld freeze?Jennifer Goodie8 Aug
        • Re: Corrupt index = mysqld freeze?Dan Nelson8 Aug
          • Re: Corrupt index = mysqld freeze?Jeremy Zawodny9 Aug
            • Re: Corrupt index = mysqld freeze?Jeremy Zawodny9 Aug
              • RE: Corrupt index = mysqld freeze?Jennifer Goodie14 Aug
                • Re: Corrupt index = mysqld freeze?Jeremy Zawodny18 Aug
  • Re: Corrupt index = mysqld freeze?Daniel Kasak8 Aug
  • Re: Corrupt index = mysqld freeze?Ask Bjørn Hansen11 Aug