List:Falcon Storage Engine« Previous MessageNext Message »
From:Kevin Lewis Date:November 3 2008 8:52pm
Subject:Re: Scavenger changes for sysbench
View as plain text  

When I tried these settings locally, I noticed an increase in the total 
CPU %.  I think is because the scavenger was basically always running, 
but I did not measure it.  I would really like to see some numbers to 
see how this affects DBT2.  If your runs are hitting the forced 
scavenges, then that is a bad thing and we should try something like 
this to avoid that.

Maybe a few selected sample high concurrency runs could help us choose 
the best setting.  Every 10 seconds?  Every 3?  I guess it depends on 
the hardware.  Also, maybe the scavenger should scavenge more per cycle 
by starting at 80% and using a floor at 50% of cache (floor=63 =50%/80%).

If the scavenger thread is not getting in the way of the client threads, 
maybe it should scavenge even less per cycle.  Maybe 90% down to 70% (of 
record cache).  This would leave more records around for possible reuse.

***  And maybe we should change the meaning of 
falcon-record-scavenge-floor to refer to the size of the record cache, 
not the size of the falcon-record-scavenge-threshold ***

Note; If you use --falcon-debug-mask=1 or any odd number this message 
will print whenever a forced record scavenge happens;
		Log::log("Forced record scavenge cycle\n");

If you want to see how many threads waited for that cycle to complete, 
change the lines above that;

	if (forced && scavengeCycle > cycle)
+		{
+		Log::log("Waited on a forced record scavenge\n");
+		}

Hakan Kuecuekyilmaz wrote:
> Kevin,
> I just read your proposal for changing the behavior of the
> scavenger for sysbench related tests:
> [quote]
>   --falcon-scavenge-schedule=1,6,11,16,21,26,31,36,41,46,51.56 * * * * *
>   --falcon-record-scavenge-threshold=80
>   --falcon-record-scavenge-floor=75
> [/quote]
> Would make sense to try
>   --falcon-record-scavenge-threshold=80
>   --falcon-record-scavenge-floor=75
> for DBT2?
> For InnoDB I got a tip from Vadim from Percona to lower
>   --innodb_max_dirty_pages_pct
> from it's default of 90% to 70% and it helped to flaten the periodic
> drops in InnoDB's performance a bit. I had a brief look into InnoDB's
> code and it looks like it flushes all dirty pages, which means that they
> probably don't have a "innodb_max_dirty_pages_floor_pct" like a thing.
> So flushing more often in smaller chunks helped InnoDB.
> Best regards,
> Hakan
Scavenger changes for sysbenchHakan Kuecuekyilmaz3 Nov
  • Re: Scavenger changes for sysbenchKevin Lewis3 Nov