On 29 Jul 2008, at 07:36, Konstantin Rozinov wrote:
> 1. It would seem that without creating another level of MySQL servers
> that would act as directories, the 1 MySQL server that would act as a
> directory in the current setup could become overwhelmed, because of
> the millions of users, comments, photos, etc that need to be looked up
> for each and every request? Is there a solution to this problem?
I agree, I think that's also a good way of introducing a nasty failure
point as well. Normally you'd use some kind of hashing function to
split data across partitions, for example put users a-m on one server,
n-z on another. That's a bit naive - you may end up with a big bias -
but you get the general idea. The thing to bear in mind is that if
your mechanism for partitioning is on the DB client end (i.e. in your
scripts on each web server), it will generally be scalable, as the DB
servers don't need to know about it, so you can keep adding them with
minimal overhead. Aside from that, I'd be tempted to think up some
scheme to keep the mapping in something faster than a DB, e.g.
memcache, though of course you'd have to have some means of rebuilding
the map in the event of cache expiry/loss. memcache can handle tens of
thousands of hits per second, and it scales so nicely.
If you have shared files (e.g. your pictures), I'd strongly suggest
you take a look at GlusterFS. It has a really nice mechanism for
handling distributed file systems with redundancy - think networked
RAID - that also scales more or less infinitely.
Synchromedia Limited: Creators of http://www.smartmessages.net/
UK resellers of info@hand CRM solutions
marcus@stripped | http://www.synchromedia.co.uk/
Attachment: [application/pkcs7-signature] smime.p7s