603f936745
framework * configure.ac: * tests/*: The tests have been reorganised into different categories: conformance, interactive and micro benchmarks. - conformance tests can be run as part of automated tests - interactive tests are basically all the existing tests - micro benchmarks focus on a single performance metric I converted the timeline tests to conformance tests and also added some tests from Neil Roberts and Ebassi. Note: currently only the conformance tests use the glib test APIs, though the micro benchmarks should too. The other change is to make the unit tests link into monolithic binaries which makes the build time for unit tests considerably faster. To deal with the extra complexity this adds to debugging individual tests I have added some sugar to the makefiles so all the tests can be run directly via a symlink and when an individual test is run this way, then a note is printed to the terminal explaining exactly how that test may be debugged using GDB. There is a convenience make rule: 'make test-report', that will run all the conformance tests and hopefully even open the results in your web browser. It skips some of the slower timeline tests, but you can run those using 'make full-report'
12 lines
1.4 KiB
Text
12 lines
1.4 KiB
Text
|
|
Outline of test categories:
|
|
|
|
The conform/ tests should be non-interactive unit-tests that verify a single feature is behaving as documented. See conform/ADDING_NEW_TESTS for more details.
|
|
|
|
The micro-bench/ tests should be focused perfomance test, ideally testing a single metric. Please never forget that these tests are synthetec and if you are using them then you understand what metric is being tested. They probably don't reflect any real world application loads and the intention is that you use these tests once you have already determined the crux of your problem and need focused feedback that your changes are indeed improving matters. There is no exit status requirements for these tests, but they should give clear feedback as to their performance. If the framerate is the feedback metric, then the test should forcibly enable FPS debugging.
|
|
|
|
The interactive/ tests are any tests whos status can not be determined without a user looking at some visual output, or providing some manual input etc. This covers most of the original Clutter tests. Ideally some of these tests will be migrated into the conformance/ directory so they can be used in automated nightly tests.
|
|
|
|
Other notes:
|
|
All tests should ideally include a detailed description in the source explaining exactly what the test is for, how the test was designed to work, and possibly a rationale for the aproach taken for testing.
|
|
|