On Fr, 2009-01-09 at 09:57 +0100, John Embretsen wrote:
> Vladislav Vaintroub wrote:
> > John, thanks for the update.
> > don't you think it makes sense to finish the MWH project and remove the
> > "falcon_team" tree hack from mysql-test-run.pl. The amount of red seen
> > recently is the team tree is alarming (for me).
> Short version: Yes, I agree.
I don't agree at all. We need a mechanism to move failing or unstable
tests in case of an upmerge. We cannot stop an upmerge just because one
test out of ~250 is failing. And no: using a different disabled.def
depending whether we are in main or Falcon trees is error prone.
Moreover, moving a test by disabling it ends up in > 20 disabled tests.
Disabled tests are not run and end up in "out of sight out mind" thus
leading to an ever increasing disabled.def list. We did that experiment
around two years ago and it failed badly, very badly. In contrary the
falcon_team test suite experiment succeeded. It took very long time -
around 9 months - but we are down from 25 or so unstable tests to 3 - 4.
I consider this as a success.
> Longer version:
> I believe the initial idea was to run the falcon_team suite only in the
> mysql-6.0-falcon-team branch (now it is also running in mysql-6.0-falcon).
> Then most of us would/should practically ignore test failures in
> falcon_team (but keep it running to be able monitor changes on all
> Pushbuild platforms), so that we can upmerge anyway. However, I don't
> think that will work very well. Red is red, and these failing tests
> could be "masking" other issues.
> I agree that the best solution would be to get rid of the falcon_team
> suite. We are well on our way to do that, as there are only 3 tests
> left, however it is still work in progress:
> - Failing (lock wait timeout instead of deadlock).
> - Hakan is looking into this one.
This fails from time to time only. I told Ann and Kevin about it. I
think Kevin is working on it. We should talk about this failure on
> - Times out with default MTR settings.
> - Not run in Pushbuild since it is --big-test.
And if run, it fails. This is
test written by Wlad and bug assigned to Chris.
> - Failing (lock wait timeout)
> - Svoj and Hakan were working in this area (e.g. Bug#34182) before the
> holidays, but I'm not sure about the current status.
Svoj fixed the falcon_deadlock issue already and is waiting for code
> I will implement the following change as soon as possible unless someone
> Remove falcon_team "hack" in mysql-test-run.pl, meaning that the
> falcon_team suite will no longer run by default in falcon branches in
I would agree on running falcon_team in mysql-6.0-falcon-team and not in
> Those working on the remaining falcon_team tests may temporarily move
> such a test into the falcon suite to see how it runs in Pushbuild. Then
> monitor results closely and revert or fix if the test fails. This should
> generally not be done right before an upmerge to main (we want to avoid
> propagating unstable tests). Alternatively, use one of the "private"
> Pushbuild branches for such testing.
I have done that in December last year already. The remaining tests in
falcon_team are unstable and to be fixed.
> Note that the falcon_team suite is also run as part of the Weekly Falcon
> Test Overview ( http://lists.mysql.com/falcon/348 ), regardless of these
Instead of complaining around which test belongs to where and how an
unstable test should be temporarily disabled, we should focus on fixing
Hakan Küçükyılmaz, Senior Software Engineer DBTG/MySQL +49 160
Sun Microsystems GmbH Sonnenallee 1, DE-85551 Kirchheim-Heimstetten
Geschaeftsfuehrer: Thomas Schroeder, Wolfang Engels, Dr. Roland Boemer
Vorsitz d. Aufs.rat.: Martin Haering HRB MUC 161028 49.011, 8.376