...
- Functionality changes not directly related to the test purpose.
- When the test are run are run.
- Race conditions.
- Environment change: This means the tests should pass on all environment variations which might be relevant for the component. Variables here could be OS, language settings, etc.
- Permissions: Tests should not assume they can change anything outside of the directory they are run in, unless explicitly configured in the test.
- Sequence of tests. Test should not be dependent on which test has run, or not run, prior to this test.
Clean output
One of the critical means of debugging the automatic test are the out generated in the applications log file and consule output. To maximize the value of these sources for runtime information, the information generated by the test should be kept separate from the application output. This includes:
- Do not commit test code with and System.out.printlns
- Use the TestLogger class for generating test log output.
- Do not log stacktraces for expected exceptions.