Integrity service

The integrity services purpose is to monitor the state of integrity on a per collection basis. This is done by periodically collecting information about files and their checksums on each pillar. Based on the collected information the state of integrity can be determined.
As a side effect, statistics about number of files and collection sizes is also generated.

A collection is considered to be consistent and in a good state of integrity if pillars all agree on the full set of files and their checksums. Therefore the state of integrity is described by:

• Missing files: Files not present on all pillars in a collection. This means that if a file is available on any number of pillars (even just a single pillar) in a collection, then it will be considered missing on the remaining pillars.
• Missing checksums: Files missing a checksum on one or more pillars. A checksum is considered missing if a pillar have reported to have the file, but have not delivered the checksum for the file.
• Obsolete checksums: Files on a pillar where the checksum have become outdated. How old a checksum can be before it is considered obsolete, depends on configuration and is specificed for a collection. As pillars use different storage techniques and media it is possible to configure specific maximum age per pillar.
• Inconsistent checksums: Files which checksums does not agree across all pillars in a collection. All known checksums for a file in a collection must agree. In case a single pillar disagrees from the remainder (two or more) pillars, it alone will be considered to have an inconsistent checksum. Else all pillars will have their file reported as having inconsistent checksum.

Integrity checks

The current service supports two types of integrity checking: full and incremental. Common for both checks is that they rely on the same database, in which they store the collected information about files and their checksums. After finished collection, the content of the database serves as a snapshot of the state of the collection and is used to determine if there are any of the mentioned integrity issues.

As the outcome of a integrity check is a report containing the (if any) found issues. Also in case of discovered issues an alarm is sent, containing a summary of the issues.

Full integrity check

The full integrity check collects the full set of file ids and checksums from the pillars and stores them in a database. Based on the freshly collected file and checksum information, the checks listed above is performed.

Incremental integrity check

The incremental check collects fileids and checksums from the point where an earlier check had collected and forward. This enables quick discovery of new or updated files, but cannot discover files no longer present, as only new or updated files will be discovered. Therefore in case of a file have been deleted, the incremental check will still report the file as present, therefore it is recommended to run the full integrity check with regular intervals.

Correcting integrity issues

The integrity service does not provide means to automatically correct issues found. The information that the service provides is intended for supporting a human being (or purposely written software) in the decision on how to act on issues.