Integrity Service tests

Workflows

Verify eksisting workflows

1. Verify that the following workflow are listed in the Integrity service configuration dropdown and the Workflow table:
   
   1. CompleteIntegrityCheck
   2. IncrementalIntegrityCheck
   3. RepairMissingFilesWorkflow
   4. SaltedChecksumWorkflow

Manually start workflow

1. Start the CompleteIntegrityCheck by clicking the Start button.
2. Verify that the current state begin to show the different step of the workflow.
3. Verify that the workflow can finish and go to the state 'idle'
4. Click on the date for the 'last run' and validate that all of the previously found steps are shown with an duration time.

Test missing file

1. Delete a file from one of the pillars (e.g. move it away from the fileDir in the ReferencePillar)
   - preferably a file with fileID starting with an alphabetically early letter as the later  RepairMissingFilesWorkflow will handle files in alphabetical order.
2. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
3. Verify that no corrupt files have been found anywhere.
4. Verify that the given pillar is missing a file (or one more file than before the running of the workflow).
5. Look at the list of existing files at the pillar, and verify that the deleted file is not amongst the list.
6. Look at the list of existing files at another pillar, and verify that the deleted file is amongst the list for that pillar.
7. Look at the list of missing files at the pillar, and verify that the deleted file is amongst the list.
8. Verify that the file is not missing at any other pillar
9. Verify that an alarm has been send to the AlarmService about the missing file. All pillars are said in the alarm message to have "<pillarid> is missing 1 file"

Restoring the missing file

1. Use the RepairMissingFilesWorkflow to restore the missing file - and wait until it is finished.
2. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
3. Verify that the file is not longer missing at the pillar.

Test corrupt file at single pillar

1. Change in the content of a file at one of the pillars (e.g. run replace-file, or add a character to it in the fileDir of the ReferencePillar) - The file must be present at all pillars initially and not have any checksum errors.
2. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
3. Verify that no pillar is missing the corrupted file.
4. Verify that only the pillar, where it has been corrupted, has the given file amongst their list of 'files with checksum error'.
5. Verify that an alarm has been sent to the AlarmService about the corrupt file.

Test corrupt file at multiple pillars

1. Choose a collection with more than 3 pillars.
2. Upload a file to all pillars
3. Run replace-file command with another file for two of the pillars (thus at least two has the original and at least two has the new file)
4. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
5. Verify that no pillar is missing the corrupted file.
6. Verify that all pillars have the given file amongst their list of 'Number of inconsistent checksums'.
7. Verify that an alarm has been send to the AlarmService about the corrupt file.

Restoring the corrupt file

1. Restore the file by running a deleteFile on the pillars with the corrupted files followed by a putFile with the original file
2. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
3. Verify that the file is no longer amongst the list of 'files with checksum error'.

Testing the Integrity Information for collection

1. Check the current integrity information about a given collection (i.e. 'Latest ingest', 'Size', and 'Number of files').
2. Upload a new file in the current measurement size of the collection information (i.e. if the size is currently measured in MB, then upload a new file of at least 1 MB).
3. Run the CompleteIntegrityCheck by clicking the Start button - and wait until it is finished.
4. Verify that the number of files has increased by 1, and the total file size has increased accordingly with the size of the new file.

Verify that workflows dates is not identical for different collections

1. Check whether the workflows have different 'last run' dates for the different collections
2. Run the CompleteIntegrityCheck for one of the collections by clicking the Start button - and wait until it is finished.
3. Verify that a new 'last run' date is updated to the time when the workflow was started.
4. Verify that the other workflows, which has not been run, are not updated by this.

Test the salted checksums integrity check

1. Run the SaltedChecksumWorkflow workflow and wait for its completion
2. Verify that the workflow was a success and that it have not resulted in sending any alarms.
   - NB: the workflow picks a random file in the collection to make a salted checksum for, so if the picked file only exists on a checksum pillar it will fail and throw an alarm - if so, just ignore it and try running the workflow again