Hakan,
When I tried these settings locally, I noticed an increase in the total
CPU %. I think is because the scavenger was basically always running,
but I did not measure it. I would really like to see some numbers to
see how this affects DBT2. If your runs are hitting the forced
scavenges, then that is a bad thing and we should try something like
this to avoid that.
Maybe a few selected sample high concurrency runs could help us choose
the best setting. Every 10 seconds? Every 3? I guess it depends on
the hardware. Also, maybe the scavenger should scavenge more per cycle
by starting at 80% and using a floor at 50% of cache (floor=63 =50%/80%).
If the scavenger thread is not getting in the way of the client threads,
maybe it should scavenge even less per cycle. Maybe 90% down to 70% (of
record cache). This would leave more records around for possible reuse.
*** And maybe we should change the meaning of
falcon-record-scavenge-floor to refer to the size of the record cache,
not the size of the falcon-record-scavenge-threshold ***
Note; If you use --falcon-debug-mask=1 or any odd number this message
will print whenever a forced record scavenge happens;
storage\falcon\Database.cpp(1785):
Log::log("Forced record scavenge cycle\n");
If you want to see how many threads waited for that cycle to complete,
change the lines above that;
if (forced && scavengeCycle > cycle)
+ {
+ Log::log("Waited on a forced record scavenge\n");
return;
+ }
Hakan Kuecuekyilmaz wrote:
> Kevin,
>
> I just read your proposal for changing the behavior of the
> scavenger for sysbench related tests:
>
> [quote]
> --falcon-scavenge-schedule=1,6,11,16,21,26,31,36,41,46,51.56 * * * * *
> --falcon-record-scavenge-threshold=80
> --falcon-record-scavenge-floor=75
> [/quote]
>
> Would make sense to try
> --falcon-record-scavenge-threshold=80
> --falcon-record-scavenge-floor=75
>
> for DBT2?
>
> For InnoDB I got a tip from Vadim from Percona to lower
> --innodb_max_dirty_pages_pct
>
> from it's default of 90% to 70% and it helped to flaten the periodic
> drops in InnoDB's performance a bit. I had a brief look into InnoDB's
> code and it looks like it flushes all dirty pages, which means that they
> probably don't have a "innodb_max_dirty_pages_floor_pct" like a thing.
> So flushing more often in smaller chunks helped InnoDB.
>
> Best regards,
>
> Hakan
