List:Falcon Storage Engine« Previous MessageNext Message »
From:Philip Stoev Date:January 29 2009 1:08pm
Subject:Re: List of weekly tests for Falcon
View as plain text  
>> * Run the combinations test suite from my "bug weekends", which is
>> essentially a series of insert/update/delete/select statements under 
>> various
>> combinations of options (such as falcon-page-size) and ratios of the
>> individual SQL statements;
>>
>
> Somewhat similar to iuds6.tst?

Iuds6 tends to do repetitive large transactions which update hundreds of 
records at a time on fairly large tables. The RQG tends to generate smaller, 
more varied transactions on smaller tables, so those tests exercise 
different workloads.

Most importantly, from my weekend testing I learned that the presence of 
certain SQL statements and their proportion influences the outcome of the 
test and bug will be triggered. For example, having DELETEs in the workload 
prevents an index from growing, causing less LIMIT crashes, even if DELETEs 
are otherwise nice to have. Therefore, I take one general SQL grammar that 
includes both updates deletes inserts and selects and execute it numerous 
times, each time masking a portion of the grammar so that certain statements 
are not generated. Once the runs are over I see which crashes happened in 
the presence of what statements and/or Falcon options in order to arrive at 
the smallest reproducible test case and estimate the impact of the bug to 
the general public.

>> * If a certain Falcon per-push RQG test is stable enough and it is
>> meaningful to run it for a longer duration, I will schedule that as well.
>> This will not include tests that do concurrent ALTERs or tablespace
>> operations, more like LIMIT stuff once LIMIT becomes sufficiently stable.
>>
>
> Although concurrent ALTER tests found a lot of bugs, I doubt the "real
> world" value of massively concurrent ALTER operations.

Yes, I think Chris has had enough concurrent alters, so I will not be 
serving any more :-)

>
> I strongly opt for LIMIT based heavy tests, because is LIMIT is
> essential in the web world use cases.

It is also important for us internally, because a broken limit also breaks 
certain unrelated tests, e.g. repeatable read (e.g. executing the same query 
with and without a LIMIT does not return the same result set, even though it 
should).

>> As noted before, it would be nice to have a weekly run of the default 
>> test
>> suites with --mysqld=--default-storage-engine=Falcon
>>
>
> Can you schedule such a test on one of your machines? Otherwise I can
> try to find a slot on lu0009 or caneland and run it on a weekly base and
> do an automated check for crashes. I am asking you to do it, because I
> think that it should be run for other engines as well and not only for
> Falcon.

I am afraid the most time-consuming part here is reviewing the test output 
and weeding out the false positives and determining valid regressions as 
compared to the previous run. I am afraid I can not be of much help there.

With respect to running the actual test, I think that editing 
include/have_innodb.inc to force innodb to appear present without actually 
using it would cause more tables and more tests to run on Falcon. Having 
such an edit would require that this test is run off a local tree that has 
this change in place. Before every run, just do "bzr pull" to bring the 
latest Falcon changes in while keeping the hacked test files.

>> Please let me know if any of this would be counter-productive and you 
>> prefer
>> a different solution.
>
> I am fine with it.

Thanks.


>
>>
>> Philip Stoev
>>
>> ----- Original Message ----- 
>> From: "Hakan Kuecuekyilmaz" <hky@stripped>
>> To: <falcon@stripped>
>> Sent: Thursday, January 29, 2009 12:23 PM
>> Subject: List of weekly tests for Falcon
>>
>>
>> > Falconers,
>> >
>> > FYI and for my Alzheimer stressed memory, here a list of tests we are
>> > running on a weekly base.
>> >
>> > Tuesday 08:45 CET
>> >  Machines: caneland, lu0009
>> >  ./mysqlslap ... --concurrency=8,64,128,192,256 ...
>> >
>> > Wednesday 08:45 CET
>> >  Machines: caneland, lu0009
>> >  iuds6.tst from system-qa
>> >
>> > Thursday 17:30 CET
>> >  Machines: esslingen, walldorf, Linux 32-bit VM, Windows 64-bit VM
>> >  WFTO, it also runs objects.objects_falcon, which creates
>> >  49166 tables and views and drops the database afterwards.
>> >
>> > Friday 08:45 CET
>> >  Machines: caneland, lu0009, nehalem-1, walldorf
>> >  DBT2 with 10,20, and 100 warehouses.
>> >
>> > Notes:
>> > nehalem-1 is out of order and crashing all the time. We will do a
>> > hardware upgrade. My estimate is 2 - 4 weeks before the machine is 
>> > ready
>> > again.
>> >
>> > Plans for other weekly runs
>> > * sysbench
>> > * mysqlbench
>> > * falcon_repeatable_read.c
>> >
>> > -- 
>> > Hakan Küçükyılmaz, Senior Software Engineer
> DBTG/MySQL +49 160 
>> > 98953296
>> > Sun Microsystems GmbH     Sonnenallee 1, DE-85551 Kirchheim-Heimstetten
>> > Geschaeftsfuehrer:  Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer
>> > Vorsitz d. Aufs.rat.: Martin Haering   HRB MUC 161028     49.011, 8.376
>> >
>> >
>> > -- 
>> > Falcon Storage Engine Mailing List
>> > For list archives: http://lists.mysql.com/falcon
>> > To unsubscribe:
>> > http://lists.mysql.com/falcon?unsub=1
>> >
>> >
>>
>>
> -- 
> Hakan Küçükyılmaz, Senior Software Engineer DBTG/MySQL +49
> 160 98953296
> Sun Microsystems GmbH     Sonnenallee 1, DE-85551 Kirchheim-Heimstetten
> Geschaeftsfuehrer:  Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer
> Vorsitz d. Aufs.rat.: Martin Haering   HRB MUC 161028     49.011, 8.376
>
> 

Thread
List of weekly tests for FalconHakan Kuecuekyilmaz29 Jan
  • Re: List of weekly tests for FalconPhilip Stoev29 Jan
    • Re: List of weekly tests for FalconHakan Kuecuekyilmaz29 Jan
  • Re: List of weekly tests for FalconPhilip Stoev29 Jan
    • Re: List of weekly tests for FalconJohn Embretsen29 Jan
  • Re: List of weekly tests for FalconPhilip Stoev29 Jan