Sometimes, the easiest way to do this is to use the file system of the linux
machine to store the files, and make reference to them in the DB...storing
not data in the DB and getting rid of all your possible problems.
From: Cabbar Duzayak [mailto:cabbar@stripped]
Sent: Saturday, July 09, 2005 12:01 AM
Subject: Storing huge amount of binary data
I will be starting a project for which I will need to store about 1 million
image files all of which are about 50-100K in size. I will be using Linux
for this project. The database won't be very busy, there will be batch image
uploads 1-2 times a day, and around 100-200 concurrent users at most, most
of which will be reading from the db and writing only session information
type of data, etc... And, I don't really need transaction support
Adding this up, the image data size will be around 50-100 Gb, and I will
need to store a total of 1-2 Gb text information (1K for each
image) along with each of these images...
First of all, I heard that Mysql does not perform very well when tablesize
goes above 1 Gb. Is this a myth? Image table is not a big deal, since I can
partition/distribute it to ~ 100-200 tables, i.e. by table_(id % 100).
However, text information needs to stay in a single table (since I need to
do queries on it for information) and there will be multiple indexes over
And, as you can imagine, I am not sure if mysql can handle something like
this, and was wondering if you can provide some feedback.
So my questions are:
1. The main question is, do you guys have any experience with this much
binary and regular data? Do you think Mysql can handle this much data in a
reliable manner (without corrupting data and/or degrading/terrible
2. Can I implement this using regular SCSI disks with regular mysql?
Or do I have need advanced solutions such as clustered, replicated, etc?
3. Again, as you can understand, I want to minimize the cost here. If you
don't think I can use mysql, do you think Microsoft SQL server is good
enough for this task?
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=1