List:Internals« Previous MessageNext Message »
From:Michael Widenius Date:March 5 2001 12:57pm
Subject:MySQL parallel server
View as plain text  

Sorry for the delayed reply; I have been on a vacation for a week and
I am now slowly catching up with all old emails.

>>>>> "Denis" == Denis Pithon <denis.pithon@stripped> writes:

Denis> Hi all !
Denis> I continue my work on MySQL Parallel Server, remember ? We have real
Denis> improvements. I describe you the changes we have done: I recall you that
Denis> we use Linux, GFS and IPVS in our config.

Denis> GFS synchronization mechanism is based on locks. MySQL seems to lock only
Denis> index file and not data file. So it handles a data file which is not
Denis> synchro-
Denis> nized with the index file. We correct this (for our use, it's not a bug in
Denis> any way!) by locking data file everywhere an index file locking occured.
Denis> We have changed:

Denis>    * sql/my_lock.c   (fcntl -> flock, GFS doesn't yet support fcntl)
Denis>    * mi_dynrec.c    \
Denis>    * mi_locking.c    \
Denis>    * mi_open.c       / In myisam/  (adds of my_lock(datafile...) calls)
Denis>    * mi_statrec.c   / 
Denis>    * There is an access (read) in mi_open before the lock is done. We 
Denis>      have moved the my_lock call before this access.

Why do you need to do this?  As MySQL will not access the data file if
the index file is not locked, there shouldn't exist a need to lock the
data file.

Denis> Now we have a MySQL Parallel Server which seems to work with the following 
Denis> options (server compiled in debug mode)
Denis>  --enable-locking --flush --one-thread --safe-mode

Denis> I could provide you patches and tests I use. I have some minor problems 
Denis> with "check" tables. I must check (no pun!) if it locks tables before
Denis> any access, and if it uses files mentionned above. If it doesn't lock,
Denis> these 
Denis> troubles are quite normal...

Check tables uses the normal MySQL locks, so this shouldn't be a problem.

Repair table is a bit different, but it may work even know...

Denis> Now, I'm working on two things:
Denis>  - mutli-threading mode
Denis>  - transactionnal tables

Denis> In mutli-threaded mode my servers freeze. But in fact, when I use a
Denis> standalone
Denis> and standard MySQL 3.23.33 server with --enable-locking the same problems
Denis> arose. Could it be some deadlocks with --enable-locking and threads ? 

I know there is some bug in lockd on Linux that when MySQL does many
locks, lockd gets confuses and mysqld hangs.  I have however never had
enough time to find out the cause of this (the external locking works
on other platform, including windows, so I am pretty sure this is a
bug in lockd).

Denis> IMHO, there is a problem in sql/my_lock.c. The lock structure is global,
Denis> there is a race condition on it but no mutex to protect its access.
Denis> Does internal locks act as mutex  for this lock structure ?
Denis> Does somebody have the same problems with enable-locking ? 

Ouch, I haven't noticed this before (old legacy code).

I don't exactly remember why the variable had to be static on some Sun
systems, but I have now moved this to a local variable and I will fix
any Sun issues when they arises.

Denis> Finally, Transactionnal tables. I'm beginning to look at BDB tables. They
Denis> use
Denis> fcntl and lock part of file (not whole files). At the moment, it's quite 
Denis> annoying for GFS which support only flock. Could I lock the whole BDB files
Denis> without having deadlock ?

Sorry, but I don't know that.  This is something that you have to
experiment with.

Denis> Last (I promise you). Are innobase tables availables for 3.23.xx series ? 

Yes; They will be in MySQL 3.23.34-max, which hopefully should be
released this week.

MySQL parallel serverDenis Pithon27 Feb
  • MySQL parallel serverMichael Widenius5 Mar
    • Re: MySQL parallel serverDenis Pithon5 Mar
      • Re: MySQL parallel serverMichael Widenius5 Mar
Re: MySQL parallel serverHeikki Tuuri8 Mar