List:Database Benchmarks« Previous MessageNext Message »
From:Peter Zaitsev Date:December 2 2005 12:14am
Subject:Re: Open request for benchmarking input
View as plain text  
On Sun, 2005-11-27 at 05:37 -0800, David Lang wrote:

> 
> http://ask.slashdot.org/askslashdot/05/11/26/0317213.shtml?tid=221&tid=4
> 
> David Lang asks: "With the release of MySQL 5.0, PostgreSQL 8.1, and the 
> flap over Oracle purchasing InnoDB, the age old question of performance is 
> coming up again. I've got some boxes that were purchased for a data 
> warehouse project that isn't going to be installed for a month or two, and 
> could probably squeeze some time in to do some benchmarks on the machines. 
> However, the question is: what should be done that's reasonably fair to 
> both MySQL and PostgreSQL? We all know that careful selection of the 
> benchmark can seriously skew the results, and I want to avoid that (in 
> fact I would consider it close to ideal if the results came out that each 
> database won in some tests). I would also not like to spend time 
> generating the benchmarks only to have the losing side accuse me of being 
> unfair. So, for both MySQL and PostgreSQL advocates, what would you like 
> to see in a series of benchmarks?
> 
> The hardware I have available is as follows:
> 
>      * 2x dual Opteron 8G ram, 2x144G 15Krpm SCSI
>      * 2x dual Opteron 8G ram, 2x72G 15Krpm SCSI
>      * 1x dual Opteron 16G ram, 2x36G 15Krpm SCSI 16x400G 7200rpm SATA
> 
> I would prefer to use Debian Sarge as the base install of the systems 
> (with custom built kernels), but would compile the databases from source 
> rather then using binary packages.

David,

The choice of benchmark depends on what kind of application would you
like to see performance for.  

Than someone speaks about one or other database to be faster than other
in general, it makes me smile.  That would be the same as tell one car
would be able to win all competitions starting from Formula-1 and ending
with off-road  racing.

There are certain well known cases when MySQL will be faster - for
example in memory storage engine is hard to beat in point selects, or
bulk inserts  in MyISAM (no transactional overhead). 

There are certain known cases when MySQL would not perform well - it is
easy to build the query using subqueries which would be horribly slow on
MySQL but decent on postgresql... but well  writing application for
MySQL you would not write such query. 


I think most database agnostic way would be to select the "workload"
from user point of view and have it implemented the most efficient way
for database in question -  for example you may find TPC-C
implementations by different vendors are a lot different. 


> 
> For my own interests, I would like to at least cover the following bases: 
> 32 bit vs 64 bit vs 64 bit kernel + 32 bit user-space; data warehouse type 
> tests (data >> memory); and web prefs test (active data RAM)

You may grab Dell DVD store:

http://linux.dell.com/dvdstore/

for Web benchmark.  It does not have PostgreSQL build in but there some
implementations available in the Internet

DBT2 by OSDL is other good candidate - it does support postgreSQL and
MySQL natively. 


If you want some raw performance number such as number selects/sec you
may use SysBench - http://sysbench.sourceforge.net


For DataWarehouse workloads you could grab  TPC-H  or   DBT3
implementation by OSDL  - We run this successfully with MySQL 

You also could take a look at http://benchw.sourceforge.net/


Let us know when you'll be testing something we'll be happy to help you
to get best results out of MySQL :)


> 
> What specific benchmarks should be run, and what other things should be 
> tested? Where should I go for assistance on tuning each database, 
> evaluating the benchmark results, and re-tuning them?"
> 
> to clarify, none of the Opterons are dual core, I have a total of 5 
> machines available to use for the tests to be both clients and servers.
> 
> 
> 
-- 
Peter Zaitsev, Senior Performance Engineer
MySQL AB, www.mysql.com

Thread
Open request for benchmarking inputDavid Lang28 Nov
  • Re: Open request for benchmarking inputPeter Zaitsev2 Dec
    • Re: Open request for benchmarking inputDavid Lang2 Dec
      • Re: Open request for benchmarking inputPeter Zaitsev3 Dec
RE: Open request for benchmarking inputDavid Lang29 Nov
Re: Open request for benchmarking inputChris Kantarjiev2 Dec