List:General Discussion« Previous MessageNext Message »
From:Adam Goldstein Date:January 28 2004 12:38am
Subject:Re: Memory Problems on G5/OSX/MySql4.0.17
View as plain text  
I have managed to get what looks like >2G for the process, but, it does  
not want to do a key_buffer of that size....

I gave it a Key_buffer of 768M and a query cache of 1024M, and it seems  
happier.. though, not noticeably faster.

[mysqld]
key_buffer   = 768M
max_allowed_packet = 8M
table_cache               = 512
sort_buffer_size         = 2M
read_buffer_size       = 2M
myisam_sort_buffer_size = 512M
thread_cache             = 8
thread_concurrency = 8
max_connections	= 1000
skip-name-resolve
skip-bdb
skip-innodb
skip-locking
ft_min_word_len= 2
join_buffer_size = 3M
query_cache_size=1024M
bulk_insert_buffer_size=256M
tmp_table_size =128M
sort_buffer         =8M
read_rnd_buffer_size=8M
record_buffer    =32M
open_files_limit=15000
tmpdir		= /tmp/		
log-bin		= /var/log/mysql/raptor-bin.log

Benchmarks are just plain weird. Here is from the linux server to the  
G5:

alter-table: Total time: 11 wallclock secs ( 0.03 usr  0.02 sys +  0.00  
cusr  0.00 csys =  0.05 CPU)
ATIS: Failed  
(output/ATIS-mysql_fast-Linux_2.4.22_21mdkenterprise_i686-cmp-mysql- 
fink-64)
big-tables: Total time: 15 wallclock secs ( 4.31 usr  2.79 sys +  0.00  
cusr  0.00 csys =  7.10 CPU)
connect: Total time: 324 wallclock secs (46.64 usr 30.27 sys +  0.00  
cusr  0.00 csys = 76.91 CPU)
create: Total time: 105 wallclock secs ( 2.04 usr  1.10 sys +  0.00  
cusr  0.00 csys =  3.14 CPU)
insert: Total time: 1237 wallclock secs (295.16 usr 73.22 sys +  0.00  
cusr  0.00 csys = 368.38 CPU)
select: Total time: 134 wallclock secs (32.39 usr  6.77 sys +  0.00  
cusr  0.00 csys = 39.16 CPU)
wisconsin: Failed  
(output/wisconsin-mysql_fast-Linux_2.4.22_21mdkenterprise_i686-cmp- 
mysql-fink-64)

and here is on the G5 locally:

alter-table: Total time: 38 wallclock secs ( 0.07 usr  0.05 sys +  0.00  
cusr  0.00 csys =  0.12 CPU)
ATIS: Total time: 20 wallclock secs ( 7.90 usr  7.77 sys +  0.00 cusr   
0.00 csys = 15.67 CPU)
big-tables: Total time: 27 wallclock secs ( 6.49 usr 16.10 sys +  0.00  
cusr  0.00 csys = 22.59 CPU)
connect: Total time: 167 wallclock secs (39.79 usr 52.78 sys +  0.00  
cusr  0.00 csys = 92.57 CPU)
create: Total time: 106 wallclock secs ( 6.12 usr  2.94 sys +  0.00  
cusr  0.00 csys =  9.06 CPU)
insert: Total time: 1257 wallclock secs (388.48 usr 311.51 sys +  0.00  
cusr  0.00 csys = 699.99 CPU)
select: Total time: 132 wallclock secs (40.22 usr 27.92 sys +  0.00  
cusr  0.00 csys = 68.14 CPU)
wisconsin: Total time:  5 wallclock secs ( 1.89 usr  1.65 sys +  0.00  
cusr  0.00 csys =  3.54 CPU)

Some of the strangeness is due to it being a live server, tested during  
low use hours. How accurate are these bench
marks, and do they represent the overall strength of the mysql server  
to handle large loads?  I can't get a good idea a to how many  
queries/sec it should be able to handle, considering I can't tell how  
complex the queries are. All I can say is the site serves 12mil  
pages/month (~100mil hits/mo), 80% concentrated into 13H of the day,  
with perhaps 40% in just 4-5hours... About 1million hits to the  
heaviest sql page/month, broken up into the above portions.

Of course, that is also not including the amount of people potentially  
trying to access the site during this time, which by eyeball estimates  
on the graphs suggest easily 25-150% more, plus the amount more that  
would come if the site could handle them and they were happy. (We all  
know web users are a fickle bunch, and will drop a slow loading site  
like a hot potato.)

-- 
Adam Goldstein
White Wolf Networks
http://whitewlf.net


On Jan 26, 2004, at 12:05 PM, Brad Eacker wrote:

> Adam,
>      Off the wall question, but is White Wolf Networks related in
> any way to White Wolf Publishing?
>
>> You may be hitting an OSX limit. While you can install more than 2GB
>> on a system, I don't think any one process is allowed to allocated
>> more than 2GB of RAM to itself. It's not a 64-bit OS yet. You should
>> be able to search the Apple website for this limit.
>
>      Could you take a look at the ulimit man page to see if it will
> allow greater than a signed 32 bit value (2G).  If it does not then
> there is still a 32 bit limitation on process size due to this basic
> constraint.
>                      Brad Eacker (beacker@stripped)
>

Thread
Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein26 Jan
  • Re: Memory Problems on G5/OSX/MySql4.0.17Brent Baisley26 Jan
    • Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein26 Jan
      • Re: Memory Problems on G5/OSX/MySql4.0.17Brent Baisley26 Jan
        • Re: Memory Problems on G5/OSX/MySql4.0.17Gabriel Ricard26 Jan
          • Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein26 Jan
            • Re: Memory Problems on G5/OSX/MySql4.0.17Brent Baisley26 Jan
              • Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein26 Jan
          • Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein26 Jan
            • Re: Memory Problems on G5/OSX/MySql4.0.17Brent Baisley27 Jan
              • Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein28 Jan
Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein28 Jan
Re: Memory Problems on G5/OSX/MySql4.0.17Adam Goldstein28 Jan