Hema,
You've done a great job with this work. The comments below have nothing to
do with the quality of your work.
> 2685 Hema Sridharan 2008-08-19
> WL#4227
> modified:
> mysql-test/suite/backup/t/backup_functions.test
Hema: Not all of the tests passed. I didn't run the falcon tests but 6
failed for various reasons on Windows. I think you need to test this on
Windows.
mysql-test-run in default mode: *** Failing the test(s):
backup_charsets.backup_triggers backup_charsets.backup_triggers
backup_charsets.backup_triggers backup_charsets.backup_triggers
backup_charsets.backup_triggers backup_charsets.backup_triggers
master.err: backup_charsets.backup_triggers: 080825 22:21:50 [ERROR]
Restore: Could not restore trigger `bup_ts`.`trau`
master.err: backup_charsets.backup_triggers: 080825 22:37:09 [ERROR]
Restore: Could not restore trigger `bup_ts`.`capu_tri`
master.err: backup_charsets.backup_triggers: 080825 22:56:16 [ERROR]
Restore: Could not restore trigger `bup_ts`.`capu_tri`
I cannot approve this patch. The time it takes to run the suite is
excessive. There are over 1200 tests run! When I ran the suite it took
almost 3 hours to run. And that was without the falcon storage engine!
I think using a brute force method to test every combination of character
sets and collations against three different storage engines and five
different basic test combinations is overkill. We need to adopt a more agile
perspective -- we should be doing partitioning.
Partitioning in this case would be examining the character sets and
determining the boundaries of the features and forming partitions on the
result. For example (and example only), we could simply partition on the
size of characters:
1-byte
2-byte
3-byte
...
We create a test run for each of these and that becomes our basis for
coverage of the capabilities of backup WRT character sets. Running all of
the character sets through all of the engines through all of the
(applicable) collations is not very efficient nor effective as at least 3 of
the tests failed on Windows under dubious conditions (reminiscent of
non-deterministic tests of the past).
If we include this suite in the pushbuild then every pushbuild machine would
slow down considerably. Given the valgrind machine's process can be measured
by the passing of days, it is unacceptable to exasperate an intolerable
performance. It goes without saying that the performance of this suite is
not good enough to push to main. Even if we enable it in the team tree, it
will mean a pushbuild run of several days. It's already bad enough at 26-30
hours!
We must avoid using brute force methods except where warranted and this type
of coverage for testing character sets and collations in backup isn't
warranted.
Chuck
