From: Kevin Lewis
Date: November 3 2008 8:52pm
Subject: Re: Scavenger changes for sysbench
List-Archive: http://lists.mysql.com/falcon/125
Message-Id: <490F648B.5040907@sun.com>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed; charset=UTF-8
Content-Transfer-Encoding: 7BIT

Hakan,

When I tried these settings locally, I noticed an increase in the total 
CPU %.  I think is because the scavenger was basically always running, 
but I did not measure it.  I would really like to see some numbers to 
see how this affects DBT2.  If your runs are hitting the forced 
scavenges, then that is a bad thing and we should try something like 
this to avoid that.

Maybe a few selected sample high concurrency runs could help us choose 
the best setting.  Every 10 seconds?  Every 3?  I guess it depends on 
the hardware.  Also, maybe the scavenger should scavenge more per cycle 
by starting at 80% and using a floor at 50% of cache (floor=63 =50%/80%).

If the scavenger thread is not getting in the way of the client threads, 
maybe it should scavenge even less per cycle.  Maybe 90% down to 70% (of 
record cache).  This would leave more records around for possible reuse.

***  And maybe we should change the meaning of 
falcon-record-scavenge-floor to refer to the size of the record cache, 
not the size of the falcon-record-scavenge-threshold ***

Note; If you use --falcon-debug-mask=1 or any odd number this message 
will print whenever a forced record scavenge happens;
storage\falcon\Database.cpp(1785):
		Log::log("Forced record scavenge cycle\n");

If you want to see how many threads waited for that cycle to complete, 
change the lines above that;

	if (forced && scavengeCycle > cycle)
+		{
+		Log::log("Waited on a forced record scavenge\n");
		return;
+		}





Hakan Kuecuekyilmaz wrote:
> Kevin,
> 
> I just read your proposal for changing the behavior of the
> scavenger for sysbench related tests:
> 
> [quote]
>   --falcon-scavenge-schedule=1,6,11,16,21,26,31,36,41,46,51.56 * * * * *
>   --falcon-record-scavenge-threshold=80
>   --falcon-record-scavenge-floor=75
> [/quote]
> 
> Would make sense to try
>   --falcon-record-scavenge-threshold=80
>   --falcon-record-scavenge-floor=75
> 
> for DBT2?
> 
> For InnoDB I got a tip from Vadim from Percona to lower
>   --innodb_max_dirty_pages_pct
> 
> from it's default of 90% to 70% and it helped to flaten the periodic
> drops in InnoDB's performance a bit. I had a brief look into InnoDB's
> code and it looks like it flushes all dirty pages, which means that they
> probably don't have a "innodb_max_dirty_pages_floor_pct" like a thing.
> So flushing more often in smaller chunks helped InnoDB.
> 
> Best regards,
> 
> Hakan