Here is a summary of the improvements to MTR test tool and suite in 5.7.4
New Tests
Added 69 new tests and enhanced several existing tests in the MTR suite.
Test Suite migration
Test suite migration activity
is continuing and in 5.7.4 we completed migration of replication suites
rpl/binlog and about 75% of main suite. Relevant WLs are:
- WL#6921 Migrate rpl suite to run with innodb engine
- WL#6922 Migrate binlog suite to run with innodb storage engine
- WL#7263 Migrate myisam specific tests
- WL#7405 Migrate partition tests in the main suite
- WL#7402 Migrate tests in main suite for federated, blackhole, merge and csv engine
- WL#7411 Migrate tests that do not show any result difference when run with innodb engine
- WL#7410 Migrate authentication tests in main suite
- WL#7404 Migrate ctype tests in the main suite
Test Suite for replication (rpl) made GTID agnostic
Global Transaction Identifiers (GTID)
was introduced in 5.6 and MTR tests were added specifically for testing
GTID. To improve test coverage we wanted to run all tests in rpl suite
with GTID ON. This was not possible because many will have result
differences and fail. We have addressed this in 5.7.4 and have added
regular regression runs of rpl suite with GTID turned on. Here is an
example command line that is run regularly:
perl mysql-test-run.pl –force –timer –debug-server –parallel=auto
–experimental=collections/default.experimental –comment=rpl_gtid-debug
–vardir=var-rpl_gtid-debug –suite=rpl –mysqld=–enforce-gtid-consistency
–mysqld=–log-slave-updates –mysqld=–gtid-mode=on
–skip-test-list=collections/disabled-gtid-on.list –big-test
–testcase-timeout=60 –suite-timeout=360
Refer
WL#7205 for more details.
Moved InnoDB compression tests to a separate suite
Tests for InnoDB compression were part of the InnoDB suite and hence
it was not easy to run only these tests with different compression
options. Compression tests are now moved to a new suite called
innodb_zip. This change is done from 5.5 onwards and following command
lines were added to regression runs.
- perl mysql-test-run.pl –vardir=var-compressed_log0
–force –big-test –comment=compressed_log0 –testcase-timeout=60
–debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=0 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1
–force –big-test –comment=compressed_log1 –testcase-timeout=60
–debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=1 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log0_level1
–force –big-test –comment=compressed_log0_level1 –testcase-timeout=60
–debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=0
–mysqld=–innodb-compression-level=1 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1_level9
–force –big-test –comment=compressed_log1_level9 –testcase-timeout=60
–debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=1
–mysqld=–innodb-compression-level=9 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log0_level9_4k
–force –big-test –comment=compressed_log0_level9_4k
–testcase-timeout=60 –debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=0
–mysqld=–innodb-compression-level=9 –mysqld=–innodb-page-size=4k
–suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1_level1_8k
–force –big-test –comment=compressed_log1_level1_8k
–testcase-timeout=60 –debug-server –parallel=auto
–experimental=collections/default.experimental
–mysqld=–innodb-log-compressed-pages=1
–mysqld=–innodb-compression-level=1 –mysqld=–innodb-page-size=8k
–suite=innodb_zip
Minor enhancement to mysql-test-run.pl
A new option “–do-test-list was added to mysql-test-run.pl . This
will take a file name as an argument and run the tests listed in the
file.
eg: perl mysql-test-run.pl do-test-list=mytests.list
$cat mytests.list
federated.federated
sys_vars.all_vars
main analyze
main archive
main blackhole
Above command will run all tests listed in mytests.list. This is
useful in development when we are working on a feature that will have
impact on tests spread across multiple suites. The relevant tests can
now be grouped together and run with a simple command line.
