List:General Discussion« Previous MessageNext Message »
From:jonathan michaels Date:July 10 1999 8:45am
Subject:Re: architectual issue
View as plain text  
On Thu, Jul 08, 1999 at 12:30:15PM -0400, Chris Newell wrote:
> Dear MySQL gods ...
> 
> I'm facing this monster architectual issue that has me banging my head
> against a wall. I'm hoping someone here can offer some help...
> 
> I'm currently writing a small collection of applications that will
> hopefully eventually need to support an *extremely* large user base.
> This user base will will initially be about 10,000 users. After a brief
> test period the user base will quickly exceed 1 million.
> 
> Here is an example of my problem...
> 
> One of the applications that we're putting together will be an address
> book of sorts. Each user will be able to enter/read/manipulate the
> contact information for as many people as they want. Here is the
> problem...
> 
> How should the tables be laid out?
> 
> What I would consider 'the right way' is impossible:
> 
> - Ideally I'd do a seperate table for each user. The problem here is
> that a) that will turn out to be a whole lot of tables. I understand
> that MySQL might not appreciate this. b) even if I could get MySQL to
> deal with it the kernel (Solaris) will only support about 32,000
> enteries in a single directory. Given that each table generates 3 files
> this would limit me to less than 11,000 users and not leave me any room
> for tables used for other apps.

it all depends on how tied to solaris and your hardware platorm 
you are.

a few years ago, i was indirectly involved in a project that 
resulted in a database (it was custom built for its purpose) 
several orders of magnitude larger than you proposed results 
here, you are talking a big wad of hardware i/o that would have 
serious i/o bandwidth issues . unless the userland was 
constrined to say serial access, that is one at a time and 
didn't mind waiting for thier data to come up on the screen.

mysql is written to takin into consideration teh limitations of 
teh structures underneath it .. unix is still unix and has a 
long way to go beofre it could take on a project of this 
magnitude and efficiently serve data in something that could 
evem remotely be called a resonable enduser time frame.

this is one of teh probelms that we encountered .. how we 
solved this and several others is not that important, the tools 
we used on the otherhand could be of some use to you in yor 
situation.

take ahike over to www.qnx.com and look att hier distributed 
data filesystem, marry this with the primitive multi threading 
in mysql (when compared to the the same class of services in 
qnx). it would take a bit of thinking, but, i can be done and 
will ouit of teh box components from teh qnx operating system.

hardware .. well a couple of intel pentium xeons with 2 mb of 
l2 cache and some fibre channel scsi's (a lot of them .. grin) 
lots of dram, nearly forgot a few gigabit ether channels to 
keep it all talking together .. this would then become a real 
cheap mainframe .. of course yo realise that your problem can 
be solved quite easily by using a normal mainframe database 
applications package and a fairly standard mainframe .. yes ?  

your problem is only a probelm for toy computers .. if its not 
a mainframe its a toy, my professor used to say.

about the only real question would be how long it would take 
(and howmuch it would cost) tp port mysql to qnx. this would 
then be a really unbeatable combination .. arguably the fastest 
and one of the 2 most reliable unix like operating systems on 
teh planet today and a relatively fast (almost complete) sql 
database. now, with all that extra performance maybe tcx would 
consider adding teh missing bits ... who knows whats around teh 
next corner.

anyway just a few thoughts from an old qnx/os9/mc6800 based data 
plunderer

regards

jonathan

ps ..

> - is there some way to talk MySQL into using more than one data
> directory? 

it would be very easy to do this in a qnx distributed 
data/processing envoronment.

-- 
===============================================================================
Jonathan Michaels
PO Box 144, Rosebery, NSW 1445 Australia
===========================================================<jon@stripped>

Thread
architectual issueChris Newell8 Jul
  • Re: architectual issueT├Ánu Samuel8 Jul
  • Re: architectual issuejonathan michaels10 Jul
  • Re: architectual issueDavid Phipps11 Jul