Unit tests

Owned by M
Last updated: Nov 28, 2011Version comment

The unit tests can be run using the Ant target "unittest". Other test suites can be run as ''ant test -Dsuite=$SUITE-CLASS

'', where SUITE-CLASS is the class name with the dk.netarkivet prefix removed. ''Eg. ant unittest'' corresponds to ant test -Dsuite=!UnitTesterSuite.

If you want to run the unit tests in another manner, e.g. from within your development environment, run the class `dk.netarkivet.tests.UnitTesterSuite` with the following java parameters:

-Xmx512m
-Ddk.netarkivet.settings.file=tests/dk/netarkivet/test-settings.xml
-Dorg.apache.commons.logging.log=org.apache.commons.logging.impl.Jdk14Logger
-Djava.util.logging.config.file=tests/dk/netarkivet/testlog.prop
-Djava.security.manager
-Djava.security.policy=tests/dk/netarkivet/test.policy
-Dorg.archive.crawler.frontier.AbstractFrontier.queue-assignment-policy=org.archive.crawler.frontier.HostnameQueueAssignmentPolicy,org.archive.crawler.frontier.IPQueueAssignmentPolicy,org.archive.crawler.frontier.BucketQueueAssignmentPolicy,org.archive.crawler.frontier.SurtAuthorityQueueAssignmentPolicy,org.archive.crawler.frontier.TopmostAssignedSurtQueueAssignmentPolicy,dk.netarkivet.harvester.harvesting.DomainnameQueueAssignmentPolicy

It is recommended to also use the option `-Ddk.netarkivet.testutils.runningAs=NONE` to avoid running unit tests that are not expected to pass.

Running Integrity-tests

To run the integrity-test it's assumed you have a running FTP server and JMS broker.

The FTP server server should grant access to a user 'jms' with password 'jms*ftp'. The user 'jms' should be able to execute the normal ftp operations like: read, write, append, list, and so forth. The FTP server should run on port 21 on localhost.

The JMS broker should be setup with a broker where username 'controlRole' and password 'R_D' is assigned and run on port 7676.

Excluded tests

We have a system for excluding unit tests from execution when they either depend on external issues or belong to code that is still under development. This prevents the unit test suite from being 'polluted' by tests that are not yet expected to work. A test can be excluded by using the `dk.netarkivet.testutils.runningAs` method:

 if (!TestUtils.runningAs("LC")) {
     return;
 }

This causes the test to end successfully unless the tests are run as the user `LC`. The tests can be run either as a specific user, as NONE or as ALL, by setting the property `dk.netarkivet.testutils.runningAs`, e.g. `Ddk.netarkivet.testutils.runningAs=LC`. When run as ALL, every unit test regardless of exclusion is run – this is used for the daily regression tests. When run as NONE, no excluded tests are run. Typically, a developer would run the unit test suite as him or herself to see ones own failing tests but not be distracted by other developers failing tests.

In order to avoid excessive exclusion, it is a good idea to generate a list of which exclusions are in place by grepping for 'testutils.runningAs' in the source. At release time, only exclusions stemming from problems that cannot be solved yet and that are not blocking the release should be in place.

Private methods

Private methods are just as deserving as public methods of being tested, but due to Java's lack of a "friend" concept, they cannot be directly accessed from other classes. Instead, we have a utility class `ReflectUtils` that provides methods for accessing private methods as well as private fields (for easier setup). An example of using reflection for tests could be:

    hc = HarvestController.getInstance();
    Field arcRepController = ReflectUtils.getPrivateField(hc.getClass(),
         "arcRepController");
    final List<File> stored = new ArrayList<File>();
    arcRepController.set(hc, new MockupJMSArcRepositoryClient(stored));
    Method uploadFiles = ReflectUtils.getPrivateMethod(hc.getClass(),
          "uploadFiles", List.class);
    uploadFiles.invoke(hc, list(TestInfo.CDX_FILE, TestInfo.ARC_FILE2);
    assertEquals("Should have exactly two files uploaded",
                 2, stored.size()); // Set as sideeffect by mockup
    ...

JUnit assertions

JUnit comes with a base package of useful assertions, but we have over time crystallized out more assertions. These all live in the `dk.netarkivet.testutils` package, which is placed together with the tests. Along with a number of miscellaneous support utilities and mock-ups (described below), there are the following new asserts in the testutils package:

ClassAsserts:: The assertions in here (`assertHasFactoryMethod`, `assertSingleton`, and `assertPrivateConstructor`) pertains mainly to singleton objects, of which there is a small handful in the system. The `assertEquals` method tests via reflection that the `equals` method obeys the requirements from `Object.equals`.
CollectionAsserts:: The `assertIteratorEquals` and `assertListEquals` methods provide more detailed messages about differences in iterators and lists than just doing `equals`. The `assertIteratorNamedEquals` is for specific use by `HarvestDefinitionDAOTester`.
FileAsserts:: These methods help inspecting test results that reside in files. The `assertFileNumberOfLines` method checks the number of lines in a file without holding the whole file in memory. The other methods are utility methods that provide more informative error messages for tests of whether files contain strings or match regular expressions.
MessageAsserts:: The one assert method here checks the generic JMS message class !NetarkivetMessage for whether a reply was successful and outputs the error message from it if not.
StringAsserts:: The three utility methods here are similar to those of `FileAsserts` in that the provide better error messages for string/regexp matching.
XmlAsserts:: These assertions help test XML structures. The `assertElementHasAttribute` and `assertElementHasNotAttribute` check for the presence of a given attribute and whether it does or does not have a given text. Similarly, the `assertNoNodeWithXPath` and `assertNodeWithXPath` methods test whether or not a node exists in a document corresponding to a particular XPath string, and the `assertNodeTextInXPath` checks if an existing node contains a specific test.

Mock-ups

As using objects in their normal contexts became more and more difficult in an increasingly complex system, we turned to [http://www.mockobjects.com/] to simplify unit tests. Additionally, we have standardized some of our most common set-up/tear-down procedures into objects of their own. An examples of how we are using this can be seen in the test class `dk.netarkivet.common.utils.StreamUtilsTester.java` and the tests found in the test package `dk.netarkivet.harvester.webinterface`.