List:Internals« Previous MessageNext Message »
From:Sasha Pachev Date:January 25 2001 4:00am
Subject:Re: parallel Mysql ?
View as plain text  
On Wednesday 24 January 2001 09:08, Mauricio Breternitz wrote:
>Sasha:
>     Replication means keeping multiple copies of the database, each
>copy being serviced by one server running in one processor. Also, each query 
>in one database is repeated on the second one. From this understanding, I 
>dont see any potential performance improvement. (I have not read the 
>replication code in MySQL yet, though). But this way
>each query would have to be done twice.

No, only each update query is repeated. For an application with a large share 
of reads, replication solves the load balancing problem quite nicely. It 
allows you, for example, to have a dedicated machine for slow statistical 
queries without tying up your machine that does live updates. You can also 
just spread your non-critical reads ( where the data does not have to be 
perfectly current - usually 1 - 2 sec delay) across many slaves, reducing the 
load on the master. There are many other ways you can do it with replication, 
depending on your needs.

>
>     Our system allows for having a single copy of the database, visible to 
>all processors.
>As you said in your note, the key is to get the locking right.

What kind of system do you have? You have not provided a good description of 
it, but from what I have read, it could be one of the following:

* a big beefy multi-processor machine

In this case, you should just install MySQL and let it do the job. MySQL is 
SMP-capable, so barring bugs in the thread library you link it against, 
everyhting will happen automagically

* a bunch of hosts with a shared disk array

In this case, you can run mysqld on each machine, with external file locking 
enabled - on by default, to turn off you do --skip-locking. Your success will 
depend on how well good your I/O to that file system is and how 
efficient/stable the locking mechanism provided by OS is. For example, it is 
really bad with NFS. You may have better luck with GFS or some similar setups 
when the hosts attach to the disk array directy or using some fast interface, 
not through a network.

>
>     Can you suggest which files I should look for placing the 
>inter-processor locks ? (I am leery of making every call to pthread_mutex
>into a global lock, since this may hinder parallelism un-necessarily).
>I think we only need to lock around opening and closing files (tables) by 
>each processor. What do you think ? (& Thanks for taking the time
>to help with this issue).

As I have said, the best thing to do at this point is to leave the code 
alone, and first try to use what is there already to see if it works for you 
to make sure you do not waste months knocking on the wrong door or 
re-inventing the wheel. If it does not, explain what it is exactly you have 
been trying to accomplish, and we will give you some suggestions on what you 
can do to make it work. If the source of MySQL needs to be modified to make 
your application work, and this is something sufficiently general to benefit 
a lot of users, we will incorporate the changes into our code base. In fact, 
in that case, we will probably do them ourselves.


-- 
MySQL Development Team
   __  ___     ___ ____  __ 
  /  |/  /_ __/ __/ __ \/ /   Sasha Pachev <sasha@stripped>
 / /|_/ / // /\ \/ /_/ / /__  MySQL AB, http://www.mysql.com/
/_/  /_/\_, /___/\___\_\___/  Provo, Utah, USA
       <___/                  
Thread
parallel Mysql ?Mauricio Breternitz23 Jan
  • Re: parallel Mysql ?Sasha Pachev23 Jan
    • Re: parallel Mysql ?Russell E Glaue23 Jan
      • Re: parallel Mysql ?Sasha Pachev24 Jan
        • RE: parallel Mysql ?Steven Roussey24 Jan
          • Re: parallel Mysql ?Jeremy D. Zawodny24 Jan
            • Re: parallel Mysql ?Michael Widenius25 Jan
          • Re: parallel Mysql ?Sasha Pachev25 Jan
        • Re: parallel Mysql ?Paul Cadach25 Jan
Re: parallel Mysql ?Mauricio Breternitz24 Jan
  • Re: parallel Mysql ?Sasha Pachev24 Jan
  • Re: parallel Mysql ?Michael Widenius25 Jan
Re: parallel Mysql ?Mauricio Breternitz24 Jan
  • Re: parallel Mysql ?Sasha Pachev25 Jan